From BrainControlClub
Jump to navigation Jump to search

Tutorial 1: Playback data, and display on monitor

We will start with the following setup wherein we will playback data to the buffer as if it is recorded in real-time. This is convenient, since it will allow you to develop and test your BCI without having to rely on real-time recordings.

Boxes depict EEGsynth modules. Orange arrows describe time-series data. Blue arrows describe Redis data

Starting the data buffer

The EEGsynth uses the FieldTrip buffer to communicate data between modules. It is the place where raw (or processed) data is stored and updated with new incoming data. For more information on the FieldTrip buffer, check the FieldTrip documentation.

  1. Navigate to the buffer module directory /eegsynth/module/buffer
  2. Copy the buffer.ini to your own ini directory (e.g. to /eegsynth/inifiles, which would be in ../../inifiles relative to the buffer module directory)
  3. Start up the buffer module, using your own ini file: ./buffer.sh -i ../../inifiles/launchcontrol.ini. Note here that the buffer module is one of the few modules that are an executable written in C, run from a bash script rather than Python. However, it does function exactly the same concerning the user-specific ini files.
  4. If the buffer module is running correctly, it does not print any feedback in the terminal. So no news is good news!

Writing pre-recorded data from HDD to the buffer

We will then write some prerecorded data into the buffer as if it was being recorded in real-time:

  1. Download some example data in .edf format. For example, from our data directory on Google Drive. Or use the data of the recording in the recording tutorial.
  2. Place the .edf file in a directory, e.g. in /eegsynth/datafiles
  3. Navigate to the playbacksignal module directory /eegsynth/module/playbacksignal
  4. Copy the playbacksignal.ini to your own ini directory (e.g. to /eegsynth/inifiles, which would be in ../../inifiles relative to the buffer module directory)
  5. Edit your playbacksignal.ini to direct the playbacksignal module to the right edf data file, e.g. under [playbacksignal] edit: file = ../../datafiles/testBipolar20170827-0.edf
  6. Edit the two playbacksignal.ini options for playback and rewind so that it will play back automatically (and not rewind): play=1 and rewind=0
  7. Make note that you can comment out (hide from the module) lines of text by adding a semicolon (;) at the beginning of the line
  8. Now start up the playbacksignal module, using your own .ini file: python playbacksignal.py -i ../../inifiles/playbacksignal.ini
  9. If all is well, the module will print out the samples that it is 'playing back'. This is that data that is successively entered into the buffer as if was just recorded

Plotting streaming data in the buffer

If you made it so far the buffer is working. However, we can now also read from the buffer and visualize the data as it comes in, using the plotsignal module. Note you need to be in a graphical environment for this.

  1. Navigate to the plotsignal module directory /eegsynth/module/plotsignal
  2. Copy the plotsignal.ini to your own ini directory (e.g. to /eegsynth/inifiles, which would be in ../../inifiles relative to the buffer module directory)
  3. Edit your plotsignal.ini to plot the first two channel, but editing under [arguments] edit: channels=1,2
  4. Now start up the plotsignal module, using your own .ini file: python plotsignal.py -i ../../inifiles/plotsignal.ini
  5. If you see your data scroll by, bravo!

Tutorial 2: using Redis for live interaction with modules

Now we have set up a basic minimal pipe-line with data transfer, we can introduce communication between modules used the Redis database. Redis is the place where modules communicate via 'key-value' pairs. Read the online documentation on the EEGsynth website for more background on the use of Redis. In this tutorial we will influence the behavior of the plotsignal output module, by changing parameters in Redis, while having the plotsignal module use these parameters as well as writing parameters back in Redis. It becomes important now to really understand the flow of information in the schema.

Boxes depict EEGsynth modules. Orange arrows describe time-series data. Blue arrows describe Redis data

Writing and reading from Redis

After installation of the EEGsynth, the Redis database should be running in the background at startup. To check whether Redis is working you can monitor Redis while adding and reading 'key-value' pairs. For the purpose of the tutorial we will use the LaunchControl MIDI controller to enter values from the LaunchControl to Redis. If you do not have a Launchcontrol, you can enter values by hand. We will discuss this as well (just skip this part).

  1. Navigate to launchcontrol module directory /eegsynth/module/launchcontrol
  2. Copy the launchcontrol.ini to your own ini directory (e.g. to /eegsynth/inifiles, which would be in ../../inifiles relative to the launchcontrol module directory)
  3. Start up the launchcontrol module, using your own ini file: python launchcontrol.py -i ../../inifiles/launchcontrol.ini
  4. You will see the connected MIDI devices printed in the terminal. If you have not set up the .ini file correctly yet, read out the MIDI device name from the output, and replace the device name, e.g. device=Launch Control under the [midi] field of your .ini file.
  5. Now restart the launchcontrol module. If everything is working correctly, a move of any of the sliders will print a key-value pair in the terminal.

You can also add values to Redis directly in Python:

  1. Start up Python, i.e. type python in the terminal
  2. Import Redis, i.e. type import r as redis
  3. Set a key-value pair, by typing r.set('test_key','10')
  4. Read a key-value pair, by typing r.set('test_key')

Patching the plotsignal module

You can now monitor the actions of Redis by opening a new terminal window and running redis-cli monitor. You should be able to see both set and get actions. So monitor this window while adding values in Redis as described above to see if it is working correctly. If you are using the launchcontrol module, you will see that the keys will be named something like launchcontrol.control077. We can tell the plotvisual module to use these values to adapt its behaviour. It will then take these values and relate them to the range of its spectral analysis, to determine frequency bands of its 'red band' and 'blue band'. The plotvisual module, in its turn, will output these frequency bands back into Redis. This makes them available to e.g. further EEG analysis. Take a moment to consider this pipeline. We call this connecting of modules via Redis parameters 'patching', referring to patching in modular synthesizers.

  1. Determine which launchcontrol sliders/rotators you want to use by moving them and looking at the values that change in Redis (use redis-cli monitor). Let's say we will do the following:
    • launchcontrol.control013 will determine the center frequency of red band
    • launchcontrol.control029 will determine the half-width of red band
    • launchcontrol.control014 will determine the center frequency of blue band
    • launchcontrol.control030 will determine the half-width of blue band
  2. Now edit your plotsignal.ini file to enter these as parameters as follows under [input]:
    • redfreq=launchcontrol.control013
    • redwidth=launchcontrol.control029
    • bluefreq=launchcontrol.control014
    • bluewidth=launchcontrol.control030

The plotsignal module will now look into Redis to try to find values there corresponding to the status of these keys. If you now change the value of any of these key-value pairs by e.g. rotating a button, the LaunchControl module will update these values in Redis, where the plotsignal module will read them and adjust its display (the red and blue lines delineating the two frequency bands). You can now move the frequency bands around, and get visual feedback overlayed on the spectrum of the channels that you are plotting. The plotsignal module also makes a conversion between the state of the values it reads from Redis (the last read position of the knobs), to frequencies in Hertz. It outputs those back into Redis, e.g. under plotsignal.redband.lo, and plotsignal.redband.hi. You can check this by using redis-cli monitor.

Tutorial 3: Real-time EEG recording with OpenBCI

Now we have set up a basic pipeline, we can replace the playback of EEG with real-time recordings of EEG. The EEGsynth supports many EEG devices, thanks to fact that we use FieldTrip's support of EEG devices in its real-time development. Read the FieldTrip realtime development documentation for more information. We only distribute OpenBCI, Jaga and GTec devices with the EEGsynth (and not all their devices yet). These modules (e.g. openbci2ft, jaga2ft and gtec2ft) are written in C, because they rely on very specific interfacing with their devices and software. However, we use them as we would any other module, i.e. start them up with a user-specified .ini file. Similarly as the playbacksignal module, these write to the buffer. They do this typically in blocks of data that are relatively small number of samples compared to the time we use to analyse the data and control devices.

Boxes depict EEGsynth modules. Orange arrows describe time-series data. Blue arrows describe Redis data

Setting up EEG recording

The most important lesson here is actually how to set up proper EEG recordings, but this falls outside of the scope of this tutorial. Please refer to our recording tutorial to familiarize yourself with the recording procedure first. When you did so, and got some experience (best is to do so under supervision of a more experienced EEG user), we will patch the EEG real-time recording in the pipeline of the previous tutorial, replacing the playbacksignal module with the openbci2ft module. If you are using another device, the principle will be the same.

  1. Navigate to openbci2ftmodule directory /eegsynth/module/openbci2ft
  2. Copy the openbci2ft.ini to your own ini directory (e.g. to /eegsynth/inifiles, which would be in ../../inifiles relative to the openbci2ftmodule directory)
  3. We will need to plug in the OpenBCI dongle into a USB port. But before you do so, do the following:
    1. Open a new terminal window and list the devices of the kernel, by typing ls /dev.
    2. Plug in the OpenBCI dongle in a USB port
    3. List the device again using ls /dev
    4. If you compare the two lists, you should see that another device was added after you plugged in the dongle. It will probably start with ttyUSB followed with a number. This is the USB port number at which the dongle is connected. If you unplug or restart your computer, this number might change, and therefor you will probably need to do this check regularly. There might be easier ways of finding the USB port number, but this, at least, is fool-proof.
  4. Edit your openbci2ft.ini file and enter the right port name for the dongle, which you can find under [General], e.g serial = /dev/ttyUSB1
  5. Start up the openbci2ft module, using your own ini file: python openbci2ft.py -i ../../inifiles/openbci2ft.ini. If things are working, you the terminal will print a message that it is waiting to connect.
  6. You can now turn on the EEG board (not the dongle) by moving the little switch to either side. After a couple of second you should see the dongle starting to blink a green and red light. This means it is configuring the EEG board with the settings specified in the .ini file, which will take a couple of seconds. After that you should have your data coming in, being transferred into the fieldtrip buffer.
  7. Now you can check the incoming data with the plotsignal module.