Finally, online experiments with E-Prime Go: Ready to go?

We're right now in the midst of the Corona pandemic and, obviously, the situation is in many ways terrible. Indeed, us E-Primers are relatively well off, given that we are not exactly at the front lines of hazardous, critical jobs or experience the economic uncertainty of freelance/precarious workers. Still, we are stuck at home as lockdowns continue, and even with the strongest measures gradually being lifted, it is still pretty much impossible to recruit human participants for studies to take place in our behavioural labs.

Online experiments may well provide a reasonable stop-gap solution and Psychology Software Tools, the developers of E-Prime, came up with a timely, free extension to E-Prime that enables us to get data from over the internet: E-Prime Go. So, seriously, a free update allows us to finally run E-Prime online?! How did they manage that? In this post, I will explain by reviewing E-Prime Go from the point of view of a typical user, answering the questions: How does it work, is it any good, what is still missing, and what should you as a user keep in mind if you want to go Online. But just to prime you with my opinion: it works locally, impressively well, though it doesn't do cross-platform, but is rather headache free and gets the job done. All in all a fantastic feat by PST that is timely, critical, and critically timed!

How does E-Prime Go work? 

What E-Prime Go does is compiling your E-Prime script file into a standalone executable (.exe) file. Specifically, it compiles the code and packs it together with a basic runtime of E-Prime, allowing you to send it by email, Dropbox, G-Drive, OneDrive, etc, to any potential participant who has a basic Windows 7 (or 8, or 10) system at their disposal. Images, videos, and necessary drivers are all automatically installed - impressively even if the participant does not have administrator rights - and the participant can then run the experiment as normal. Following completion, a popup appears with the data files and, optionally, system information file, after which the participant is to go through the motions of attaching these files to an email or upload them to a linked server. 

Specifically, let's run through a conversion from a normal E-Prime experiments to an online, E-Prime Go one. 

1) After downloading E-Prime Go (go to pstnet's support, login, download), installing it, and running it, you will be greeted by the following screen:

E-Prime Go

2) To the top, fill in the Experiment. Clicking browser, you are requested to direct E-Prime Go to the experiment script - that is to say, the uncompiled script or .ebs3 file (the 'full script' in E-Studio), not the E-Studio .es3 file. 

3) The text fields are then enabled. These are pretty straightforward: start instructions could indicate some pre-experiment instructions (like: exit all other running applications), stop instructions follow the experiment and may tell the participant how to send the data (see 4), failure message is what pops up if there's an issue during the experiment, and disclaimer is note on legality, defaulting to a warning that participants should sign informed consents in some way. Somewhat counter-intuitively, the disclaimer to the bottom is shown first to the subject, so consider that as you make changes to the text. 

4) After pressing pack and waiting for a bit E-Prime Go usually says presto and presents you with a single executable. Run it to know what your participants will see, which will basically be the disclaimer:

Disclaimer

followed by the instructions:

Instructions

(mine being much longer)

The subject can then run the task. It takes a few seconds to load, but then we're pretty much in, and everything should look very much like it does in the lab - no window borders or easy way for participants to close the experiment prematurely:

One of my experiments, now in E-Prime Go!

After finishing the task successfully, a popup is shown:

After running E-Prime Go

To the top is the text you filled in. Clicking on the lower link, I'm forwarded to the location on my filesystem containing a .zip archive with the data from the participant. This zip is what your participant will need to email to you (or to upload to some fileserver, such as OneDrive/Dropbox/GoogleDrive). If you look inside it, as i did to the bottom right in the screenshot above, you will see a bunch of files: advisor.xsl (the E-Prime advisor's sage notions on correct timing), the .startupInfo3 (containing info on the E-Prime Go version used), the actual data files (.edat3 and the plain .txt file), and the machine info as collected by DirectX (windows type, video card, and a lot more). 

And that's that. It's quick, seems reliable enough, gets the job done. 

Is it any good?

If you know Psychopy or OpenSesame, you may be aware that both provide server-based solutions to doing online experiments (respectively Pavlovia and JATOS). In comparison, E-Prime Go is a little bit clunky. For example, forget about compatibility with Android or OSX, E-Prime Go is only Windows and does not run from the browser. Instead, the local PC provides the timing, rather than the server computing things for the participant, which may mean your participant need more computing power and timing may depend more on the PC configuration of the participant. Following running your experiment, you will also have to depend on a participant's cooperation and computer skills to actually send you the data. Realistically speaking, I would give it about 50% chance for my more computer illiterate uncle to send me a completely random file, if I would even get him to go through the process of running an executable straight from the web *but note UPDATE 2 below!*. On the other hand, the experience is much less headache inducing than having to rewrite code in Javascript or having to find out what packages work and which ones don't. And I feel like a bit of a newbie here, but I am not really looking forward to spending time learning how to set up a server only to find out that my particular experiment won't run. Instead, E-Prime Go just uses a local runtime for the E-Prime script, which means that generally speaking, if your experiment works in E-Studio, it will work on your participant's PC also (provided it is not ancient and runs Windows). PC configurations can, of course, be radically different across PCs, but this is not necessarily obviated by having a server-based solution: If my participant gets a Skype call in the middle of the experiment, it will confuse them this way or that. Furthermore, E-Prime Go allows you to submit the system information along with the experimental data, so you can always figure out later whether some participants systems were too slow or one running too many background applications. 

What is still missing?

Given that PST delivered this extension with very little notice, I would say it's undeniably an impressive feat. However, the amount of haste in putting this out there feels somewhat noticeable at times. I am told that E-Prime Go is still very much a work in progress, and even as it already 'does the job', it will still get polishing and features. Hopefully, by the time you read this then, my small list of feature requests may already be implemented. 

One noticeable feature still missing from E-Prime Go is a solid way of error handling. For example, the picture on the right is a screenshot sent by my brother who tried to run my experiment, which sadly crashed about halfway through. Good: it gives some debug info: "Factor Error: No such attribute xpos251", which is good enough to identify for me where the experiment went wrong: it couldn't find attribute xpos251. Bad: what line number did it crash? (no, it's not 15,005, even my code isn't that verbose). Worse: where is the normal .txt output we can usually perform e-Recovery on? Once an E-Prime experiment is successfully completed, participants will get a pop-up with the data, but if there's any issue, then we're having much more loss than is usually the case. The present error was caused by my own poor coding, of course, but if we have to deal with all sorts of random PCs, then surely unexpected crashes can be expected. 

Minor features missing and requested:

• Right now, E-Prime Go always starts with default values. It would be nice if they would at least be retained from the previous session, because filling them in every time gets tiring. Even better would be if the startup and finishing information would be saved to, and automatically filled in from, a text-file with same name as the E-Prime script (.ebs3) file, so you can keep on making changes to multiple experiments. 
• Participant numbers and session numbers are suggested to be given in the startup info, but it would be more convenient to just have a new session number for every participant. I would appreciate the ability to rename session/IDs following experiment to improve privacy and reduce the chance they overwrite previous same PP/session combinations (this has happened too often before!)
• The final instructions should enable us to present one clickable link for the experiment data, and one for the location of where to upload data, but it is not the most elegant solution. It would be fantastic if PST could setup a tiny server which allows participants to drag/drop their data, although in the meantime, we can always create Google Forms for something like this. *but note UPDATE 2 below!*

What should you keep in mind while developing online E-Prime experiments?

• Don't forget to add the 'macro' @folder to the stop instructions otherwise your participants will have a hard time finding the files. In fact, at the moment it is a little bit too easy to simply close the window E-Prime Go finishes with, at which point the participant may wonder what happens to their data. Remind them: their data are not automatically uploaded. 
• Keep in mind that screens differ radically across subjects. I would recommend using E-Prime 'set display to desktop resolution' as there's little way of predicting what different people have at their disposal. That also means your experiment needs to scale images and use percentages rather than absolute values. 
• External communication in your experiment, like EEG trigger writing using serial/parallel port may work on your home desktop computer even without the connection physically being present, but not necessarily some random one out there. Remove unused devices, such as sound devices, parallel ports. 
• Are you using sounds? Keep in mind that you will need to instruct participants to set their speakers or headphones to a reasonable level. You could also develop your experiment to start with a climbing procedure, and then set that level at the global level that is then referenced every time you play a sound. 
• Participants don't always get instructions and usually in the lab you can fix that easily by asking them to relay any questions to the experimenter, but online there's little control over such matters. I find it useful to include a little video I made from screencaptures (I use the Windows 10 gamebar to record these with E-Prime running in Windowed mode). 

Was this post useful?

If you liked what I wrote, maybe you will consider doing my experiment? Way to drop that guilt trip on you after reading what you thought was just an informative freebie! If you were forewarned about the cost involved, would you even have started reading? Well, you can always pretend you lost interest in this post just as you finished the above paragraph. Or maybe you can do me a favour and do a little study? As a bonus, you get to witness E-Prime Go in action!

So here's a motor-timing task, it takes about 40 minutes, and is as boring as most behavioural experiments out there, but I would appreciate if you do it anyway. It requires a mouse (do not use the touch display if there is one) or similar pointing device (touchpad, trackball) and a Windows 7+ OS. 

Get the experiment here, send data using this form, or to my email or upload it to the same place you got the experiment from. There's also a .zip file if you are cannot save and directly run executable files, use this link, but first extract the file (e.g. to desktop) and then run it from its new location rather than running it straight from WinRar/7zip or what have you. Many thanks!

UPDATE 1: So will your experiment run on your computer?

On how many computers does it run then? Will sound play? Video? I have very little data myself, but I designed a little technical test that takes literally 20 seconds and will at least tell you whether E-Prime Go experiments run on *your* computer. Running this on your PC will tell you. There's also a .zip file if for good reason you are unable to save executable files or get confused by the many warnings, see here. However, this may mean you will first have to extract the files (running it straight from winrar, for example, does not seem to work).

UPDATE 2: What's next?

As I mentioned, E-Prime Go is still somewhat of a work in progress. In particular, it seems they are currently working on making it easier to distribute and collect data, so maybe my uncle will be able to participate after all. I will update this blog post as soon as I have the information.