a comparison of training methods
¹Fabrication Lab, Harvard University Graduate School of Design; e-mail: [email protected]; ORCID: 0000-0002-6640-2745
Effective safety training of users is an essential part of safe and proper operation of any maker space. This paper will assess the relative effectiveness of in-person and online safety training for laser cutting by analyzing the recorded safety incidents in the Fabrication Lab at Harvard University's Graduate School of Design (GSD). Replacing in person trainings with online equivalents provides students/users access to training at their leisure and frees up limited staff resources and time to engage and address other needs of their space. The information provided in this paper is intended to help managers and staff of maker spaces in an academic settings better allocate their resources and time by highlighting that positive outcomes online trainings has afforded the GSD.
The Fabrication Lab at the GSD provides a well equipped workspace with woodworking tools, metal working tools, CNC equipment, 3d printers and laser cutters. This paper will focus on the laser cutters at the GSD, a resource in which over 500 students utilize each semester (over 50 percent of the enrolled student population). The users of this tool are primarily graduate students enrolled in professional or post-professional degree programs in architecture, landscape architecture, urban planning, urban design, design, engineering, and design studies.  The lasers are available 24 hours a day 7 days a week to any user who has completed required training. During the academic year, the lasers are staffed by student technical assistants 8-12 hours a day. This leaves up to 16 hours of unsupervised laser access for users.
Data was collected each semester starting with the Fall 2015 semester and concludes in the Spring 2019 semester. Over this time period, there were up to nine laser cutters available to students. The maximum output power of each laser ranged from 60w to 150w (three 60-watt lasers, four 120-watt lasers, two 150-watt lasers). This laser system and setup was designed using engineering controls to reduce users’ exposure to hazards that may occur during laser use . The hazards this configuration mitigates include exposure to the laser beam and the combustion byproducts created during the laser cutting process. Engineering controls to mitigate hazards were implemented with a single start-stop switch. When activated by the user, compressed air is supplied to the laser optics, and initiates the exhaust system and supply power for the laser cutter. The lasers are CO2 lasers in a Class I enclosure. However, one hazard that could not be mitigated fully with engineering controls is fire. Occurrences of fires can only be mitigated by administrative controls (training). This study will use recorded fire events as a metric to evaluate training.
Usage data was captured using Papercut NG. This software records the number of unique users who use the laser cutters and the number of unique jobs sent to the laser cutters. This is done by requiring users to enter their username and password for each laser job they send. Data on fire events was collected through emails from students who self-report fires, end-of-shift reports from technical assistants that indicated a fire had occurred, and maintenance logs indicating fire as cause for required maintenance. Comparing these records filtered out duplicate reports, resulting in only one report per incident being used.
In Fall 2015, Spring 2016, Fall 2016, and Spring 2017 training for the laser cutters was done in-person. This consisted of a one-hour training session during which users would come to the laser cutting room of the Fabrication Lab and attend in-person training. The in-person training outline  details the content covered. The content was covered by giving a demonstration of laser operation, identification of fire extinguishers in the room, powering on the machine and sending a file that was laser cut. These training sessions were primarily offered during the first two weeks of the semester during normal business hours. Additional sessions were conducted during the semester when enough demand necessitated it and normal lab operations and access to trained users would not be disrupted. Time constraints, equipment constraints and necessary throughput to accomplish trainings in a specified time without disrupting normal lab operations limited the interaction of users with equipment during training sessions.
In response to the time and resource constraints that existed with in-person trainings, all training were then presented in an online format. This enabled users to access training at a time convenient to them and freed staff to focus time and effort on other areas of the Fabrication Lab. All users who had previously completed training in-person were not required to take the new online training.
In Fall 2017, Spring 2018, Fall 2018, and Spring 2019 training for the laser cutters was offered as an online training that had no requisite in-person component . Users who had completed training prior to the online training retained their ability to use the laser cutters without needing to complete the online training. Additionally, users are only required to complete training once during the course of their studies unless they have demonstrated a flagrant disregard of policies. The online training presented users content from the in-person training outline, formatted for online delivery using the Articulate Storyline software into a series of slides followed by multiple choice questions, and delivered through Harvard University's Center for Workplace Development online training portal (branded as the Harvard Training Portal). Portions of the in person training that required a physical demonstration, interaction or identification of physical objects such as fire extinguishers, where presented using webgl technologies as interactive three dimensional environments. All talking points and policies presented in the in-person training were presented as a series of slides utilizing images and text.
The training that evaluated parts of the workflow that could result in a fire incident on the laser were evaluated using a simulated environment. These were parts specifying laser settings (power, speed, ppi)  and focusing of the laser . This allows for direct user interaction with the portion of the laser cutting workflow that if done incorrectly, could result in a fire. This interaction was assessed for competency through the online training portal. This is significantly different than the in-person training which only allowed for users to observe the workflow without direct interaction.
Online training can be completed or repeated by users anytime during the semester from a computer with an internet connection. The online trainings are self-paced meaning users can stop the training and resume at their convenience. Data from the Harvard Training Portal indicates it takes users an average of 30 minutes to complete all training required for laser cutter access.
Overview of Data Collected
The data evaluated was collected beginning in the Fall 2015 semester and concludes at the end of the Spring 2019 semester. The number of recorded fires is likely less than actual fires that occurred. Fires that occurred after hours, when staff is not present, rely primarily on users self reporting. This likely resulted in fires not being reported that didn’t cause any obvious damage that would be noticed by staff.
Unique users refers to the number of individual users who utilized the laser cutters. Each user was counted once each semester regardless of the number of times they used the laser cutters. Unique jobs refers to the number of uniquely identifiable jobs sent through Papercut to the laser each semester. Copies cut from the same job file were counted as ‘1’ unique job.
Over the eight semesters of data collected the average number of unique users is 555 per semester. The average number of unique jobs sent is 35829 per semester. The average number of fires is 2.5 per semester. This resulted in one fire per 222 users and one fire per 14,331 jobs sent over the course of the data collection period (see table 1 & 2) . On average there were 10 more users who laser cut each semester before the implementation of online training. Overall, there was one fire per 160 users and one fire per 11589 jobs sent when training was done all in-person, and there was one fire per 366.6 users and one fire per 20729.5 jobs sent when training was introduced in an online environment.
From the data we can see that there were more fires per user and jobs sent in the semesters prior to the implementation of online training, with the high in Fall 2016 of seven documented fires (see table 1).
There is a reduction in users and jobs sent to the lasers after the implementation of online training. Anecdotally, this correlates with an increase in 3d printer usage at the GSD, which is not documented here. The information here does not evaluate whether there is a correlation between a reduction in use and a reduction in number of reported fires, only that there were less fires recorded after online training was implemented.
The data suggests that online training is no less effective in preventing incidents of fires on the laser cutter than in-person training. This conclusion is not definitive since there are a multitude of variables that can be evaluated in addition to the data collected in this paper.
This paper does not review how the institutional ‘culture’ affects the behavior of users and safe behavior. As a companion of adequate training, a culture of safety  has a significant impact on user behavior and incidents. The culture of safety at the school was not evaluated or measured during the time data was collected. In addition, information on the materials being cut that were part of a fire incident were not recorded. Investigating this may lead to information about users being unable to work safely with certain materials.
Although not evaluated in this paper, it is suggested that simulated training may also effectively train users in operation of equipment. Simulations have successfully been used in healthcare for procedural processes .
In this paper we examined the number of recorded fires each academic semester that occurred on the laser cutters in the GSD’s Fabrication Lab. These recorded instances were compared against the type of training available to users during the semester and drew conclusions about the effectiveness of training delivery methods. Beyond the type of training the users received, we did not evaluate other variables that exist such as laser cutter power, materials being cut, degree program of users, time of day of fire or other potential variables that could have an effect on the prevalence of fires that result from laser cutting. The information presented in this paper suggests that Academic Maker Spaces can implement online training that are no less effective in preventing fires than in-person training. By presenting training in an online format, time and human resources can be freed up for other tasks. As such, relieving the workload and pressure on existing staff by making training accessible remotely and simulating processes  will enable focus on tasks and duties that may otherwise be overlooked or addressed inadequately.
Table 1: Fire, user, job, and laser count by semester
# of recorded fires
# of lasers
# of unique users
# of unique jobs
**introduction of 120 desktop 3d printers at the GSD
Table 2: fire per user & fire per jobs sent
* These semesters had 0 fires, value denotes total number of users or jobs sent without fire
# unique users per fire
# unique jobs sent per fire
In person laser training checklist
Online laser training content
Screenshots from online interactive laser training
Laser setting exercise - https://fablabgsd.s3.amazonaws.com/lasertraining/setting/index.html
Laser focusing exercise -https://fablabgsd.s3.amazonaws.com/lasertraining/focus_ui/laser/focus.html