Saturday, July 4, 2015

Flappy Brain - Cambridge Jam Raspberry Pi project

At the last Egham Raspberry Jam I showed the Mindflex game EEG Brainwave reader (CNET article from when it was launched : http://www.cnet.com/uk/news/moving-objects-with-mattels-brainwave-reading-mindflex/) displaying bar graphs for the different brainwave signals.  It was interesting but served no real purpose other than to show the EEG was doing something. Even got a mention on the Linux Luddites Podcast (https://linuxluddites.com/shows/episode-41/)


So, for the Cambridge Raspberry Jam I wanted to get the EEG reader to be used for something.  I decided it had to be a single signal and immediately Flappy Birds came to mind.
Nice simple one button game.
Decision made I went about coding the game in Python on the Raspberry Pi.  I had already done the work to get the EEG

To hack the Mindflex and get it working with an Arduino I used this great guide and the code from the following article: http://frontiernerds.com/brain-hack.
This articles gives a lot of detailed background on the Mindflex and how it as well as lovely clear pictures showing where the TX and GND wires need to be attached.
For strain relief the article uses hot glue.  I wrapped the wires around the already in place knot so the wires could not be pulled no matter what.

The Processing code provided didn't work for me on the Raspberry Pi and that's the main reason I had to code up my own visualisation program.




The example I used to send the data is BrainSerialTest with one small modification. I commented out the following line as it caused the import and breaking out of the variable in Python to fail.  (Limit of my coding ability showing):
        Serial.println(brain.readErrors());

To read this from Python I had to install pyserial as it wasn't installed by default using the following commands.

apt-get update
apt-get install python-serial

You will have to use sudo for this. I left it out from the commands so that  you have to make the decision to run the commands

With the Mindflex hacked, and hooked to the Arduino and then sending it's data over serial I did the code to visualise the data.  Here is the Python code  I used on the Raspberry Pi for the last Egham Raspberry Jam

import serial
import pygame
# initialise pygame
pygame.init()

# Define the colors we will use in RGB format
black = [ 0, 0, 0]
white = [255,255,255]
red = [255, 0, 0]
blue = [0,0,255]
darkBlue = [0,0,128]
pink = [255,200,255]
green = [0,255,0]
orange = [255,102,0]
brown = [153,102,0]
yellow = [255,255,0]
purple = [128,0,128]

# Set the font for the text. Windows computer so usd Ariel

myfont = pygame.font.SysFont("OpenDyslexic", 18)


# Set the height and width of the screen
size=[800,600]
screen=pygame.display.set_mode(size)
# Fill the screen White
screen.fill(white)
# Put something in the application Bar
pygame.display.set_caption("Serial Mind reader")
pygame.display.update()

done = False
div = 2000
fatt = 20
fmed = 20
fdelta = 20
ftheta = 20
flalpha = 20
fhalpha = 20
flbeta = 20
fhbeta = 20
flgamma = 20
fhgamma = 20

ser = serial.Serial('/dev/ttyUSB0',9600)

# Clear the serial input buffer
ser.flushInput()

pygame.draw.rect(screen, white, (0,0,800,600), 0)
pygame.display.update()


while done == False:
    for event in pygame.event.get():
        if event.type == pygame.QUIT:
            done=True

    data = ser.readline()

    signal,att,med,delta,theta,lalpha,halpha,lbeta,hbeta,lgamma,hgamma = data.split(",")

    print signal
    print att
    print data


#  clear screen before putting new square on it 
    pygame.draw.rect(screen, white, (0,0,800,600), 0)
    pygame.display.update()


    fatt = int(att)
    fmed = int(med)
    fdelta = int(delta)
    ftheta = int(theta)
    flalpha = int(lalpha)
    fhalpha = int(halpha)
    flbeta = int(lbeta)
    fhbeta = int(hbeta)
    flgamma = int(lgamma)
    fhgamma = int(hgamma)

    pygame.draw.rect(screen, red, (1,600-(fatt*4),75,fatt*4), 0)
    pygame.draw.rect(screen, blue, (76,600-(fmed*4),75,fmed*4), 0)
    pygame.draw.rect(screen, pink, (152,600-(fdelta/div/2),75,(fdelta/div/2)), 0)
    pygame.draw.rect(screen, green, (227,600-(ftheta/div),75,(ftheta/div)), 0)
    pygame.draw.rect(screen, orange, (303,600-(flalpha/div),75,(flalpha/div)), 0)
    pygame.draw.rect(screen, darkBlue, (379,600-(fhalpha/div),75,(fhalpha/div)), 0)
    pygame.draw.rect(screen, brown, (455,600-(flbeta/div),75,(flbeta/div)), 0)
    pygame.draw.rect(screen, black, (531,600-(fhbeta/div),75,(fhbeta/div)), 0)
    pygame.draw.rect(screen, yellow, (607,600-(flgamma/div),75,(flgamma/div)), 0)
    pygame.draw.rect(screen, purple, (683,600-(fhgamma/div),75,(fhgamma/div)), 0)

    screen.blit(myfont.render("Attention",1,black), (4, 1))
    screen.blit(myfont.render("Mediation",1,black), (79, 1))
    screen.blit(myfont.render("Delta",1,black), (155, 1))
    screen.blit(myfont.render("Theta",1,black), (230, 1))
    screen.blit(myfont.render("Low Alpha",1,black), (306, 1))
    screen.blit(myfont.render("High Alpha",1,black), (382, 1))
    screen.blit(myfont.render("Low Beta",1,black), (458, 1))
    screen.blit(myfont.render("High Beta",1,black), (534, 1))
    screen.blit(myfont.render("Low Gamma",1,black), (610, 1))
    screen.blit(myfont.render("High Gamma",1,black), (686, 1))
# show the whole thing
    pygame.display.update()

pygame.quit ()


As a side note. I did a D&T day at my kids school where I used this code for a Year 1 demonstration. All the kids got to see their brainwaves bouncing around as they moves, did maths and recited poems.  It even went really flat when the kids closed their eyes and calmed down.   Some of the kids had to wait so long that they missed their morning break for the opportunity to see their brains waves bouncing around,

Next onto getting the Flappy Birds inspired game (for the purest you could say helicopter inspired) game working.
I thought myself to code in the 80's on a Commodore 64 and to be honest I haven't updated my skills since, so doing the same in Python was a new challenge for me.
Fortunately, I read an article in Linux Format Issue 112 from December 2008 that made a top down racer using Python and Pygame in 100 lines of code. http://www.linuxformat.com/includes/download.php?PDF=LXF112.tut_code.pdf
Yes, using a tutorial from over 7 years ago.  I may not be great at coding but I do hoard useful bits of knowledge.  I bought the Mindflex from eBay in September 2014 and did nothing with it until January.  Another bit of my electronics stash.

The Racer game if you think about it is the same but more complex than the Flappy Brain game with a few differences.
Racer scrolls vertically, Flappy  Brain scrolls horizontally.  That's just changing the value of the X variable instead of the Y variable
Then it's just change some of the graphics.

Once I figured out the Racer code I worked out the bits I could throw away and then bits I had to keep as well as worked out how to modify the code to do what I needed.

This took me a while to work through again due to a lack of knowledge or practice.

In the end I created the final game that was shown at the Cambridge Raspberry Jam.

Code Below:

from pygame import *
import random
import time
import serial

# set up the serial port
ser = serial.Serial('/dev/ttyUSB0',9600)

# Clear the serial input buffer
ser.flushInput()

# variables for colours 
black = [ 0, 0, 0]
white = [255,255,255]
red = [255, 0, 0]
blue = [0,0,255]
darkBlue = [0,0,128]
pink = [255,200,255]
green = [0,255,0]
orange = [255,102,0]
brown = [153,102,0]
yellow = [255,255,0]
purple = [128,0,128]

# gap in wall
gap  = 200

# width and height of screen 
width = 1000
height = 600
count_gap = int(height/gap)

# 0 = hard, 1 = easier, 2 is easiest
difficulty = 2

# class to create sprites and render them
class Sprite:
    def __init__(self,xpos,ypos,filename):
        self.x = xpos
        self.y = ypos
        self.bitmap = image.load(filename)
    def render(self):
        screen.blit(self.bitmap,(self.x,self.y))

# screen size
size=[width,height]
# initialise pygame
init()

# create the screen (window)
screen = display.set_mode(size)

# Caption to go at the top of the window
display.set_caption("Flappy EEG 2")

# set the music, its volume and start to play. -1 means infinite loop
mixer.music.load("Pinky_and_the_Brain.ogg")
mixer.music.set_volume(0.5)
mixer.music.play(-1)

# crreate sound for crash
crasheffect = mixer.Sound("ouch.ogg")


# fill the screen with blue, update the screen
# not really required but I like it
screen.fill(blue)
display.update()
#time.sleep(1)

# create the sprites for the brain, columns and the background image

playerbrain = Sprite(20,200,"brain_75.png")
column1 = Sprite(1200,100,"column1.png")
column2 = Sprite(1200,100,"column2.png")
background = Sprite(0,0,"background.png")

# set fonts for different purposes
scorefont = font.Font(None,60)
datafont = font.Font(None,20)
overfont = font.Font(None,100)

# set default values for some variables
score = 0
maxscore = 0
quit = 0
gameover = 0

# master loop for the program. If quit == 0 then exit program
while quit == 0:
# flush the serial port of all data to begin fresh
    ser.flushInput()

    gameover = 0

# set the height of top column
    column1.y = (random.randrange(0,(count_gap))*gap)-800

# set the height of the bottom column
    column2.y = column1.y + 800 + gap

# start of loop (using while) to move the columns

# start off screen to the right
    x = width +50

# x>-100 means still valid (yes there is a minus in there)
# gameover when collision
# quit selected. either pressed q or click x on window
    while x >-100 and gameover == 0 and quit == 0:

# increment the score and if higher than maxscore make maxscore = score
        score = score + 1
        if score > maxscore:
            maxscore = score

# update columns location and set x positions
        x = x - 50
        column1.x = x
        column2.x =x  

        data = ser.readline()
#        print data

        signal,att,med,delta,theta,lalpha,halpha,lbeta,hbeta,lgamma,hgamma = data.split(",")

        print "signal: " + signal
        print "attention: " + att
        print "data: " + data
        intatt = int(att)
        if intatt > 90:
            intatt = 90   
        brainpos = intatt * 6
# set brain location based att (attention)

# is intatt near the gap above 
        if brainpos < column1.y +800 and brainpos > column1.y + 800 - (difficulty * 10):
            playerbrain.y = column1.y +800 +70
            print "brain near top and moved down!"
# is intatt near gap bottom
        elif brainpos > column2.y-75 and brainpos < column2.y + (difficulty * 10):
            playerbrain.y = column1.y +800 +70
            print "brain near bottom and moved up!"

        else:
            playerbrain.y = brainpos
            print "brain where is should be"


#        print playerbrain.y
        background.render()
        playerbrain.render()
        column1.render()
        column2.render() 

# display some information on screen
        screen.blit(scorefont.render("Score: "+ str(score),1,white), (100, 5))
        screen.blit(scorefont.render("High Score: "+ str(maxscore),1,white), (400, 5))

        screen.blit(datafont.render("signal: "+ signal,1,white), (5, 570))
        screen.blit(datafont.render("attention: "+ att,1,white), (150, 570))

        screen.blit(datafont.render("playerbrain.y: "+ str(brainpos),1,white), (250, 570))
        screen.blit(datafont.render("column1.y: "+ str(column1.y+800),1,white), (500, 570))
        screen.blit(datafont.render("difficulty: "+ str(difficulty),1,white), (650, 570))

        display.update()

#        print playerbrain.y

# collision dection
        if ((playerbrain.y < column1.y+801 or playerbrain.y > column2.y-74) and (x <150 and x > 20)):
            mixer.music.stop()
            mixer.Sound.play(crasheffect)
            print "BUMP"
            gameover = 1

# check if clicked x on window to exit 
        for ourevent in event.get():
            if ourevent.type == QUIT:
                quit = 1

# has key been pressed. K_q is to quit 
            if ourevent.type == KEYDOWN:
                if ourevent.key == K_DOWN:
                    playerbrain.y = playerbrain.y+10

                if ourevent.key == K_UP:
                    playerbrain.y = playerbrain.y-10

                if ourevent.key == K_q:
                    quit = 1 

# if game over show message
    while gameover == 1 and quit == 0:
        screen.blit(overfont.render("GAME OVER",1,yellow), (380, 260))
        display.update()

# then wait for a key to pressed  before starting again
        for ourevent in event.get():
            if ourevent.type == KEYDOWN:
                if ourevent.key == K_0:
                    difficulty = 0
                    score = 0
                    gameover = 0
                    mixer.music.play(-1)

                if ourevent.key == K_1:
                    difficulty = 1
                    score = 0
                    gameover = 0
                    mixer.music.play(-1)

                if ourevent.key == K_2:
                    difficulty = 2
                    score = 0
                    gameover = 0
                    mixer.music.play(-1)

                if ourevent.key == K_SPACE:
                    score = 0
                    gameover = 0
                    mixer.music.play(-1)

                if ourevent.key == K_q:
                    quit = 1 
                    score = 0
                    gameover = 0


The sharp eyed amongst you will recognise the bits taken from the Racer game and the bits taken from the original visualisation code I create.

I added the ability to Quit the game by pressing 'q' and also 3 difficulty levels. On the Game Over screen press 1, 2, or 3 with the higher number being easier.
It does this by widening the readings that will get you through the gap.

At the Cambridge Raspberry Jam I had the pleasure of meeting Alex Eames who brought us to HDMIPi and also the RasPi IO Duino which was another successful Kickstarter and is now for general sale.  Alex did a great video which you can see below.



Following this the project was picked up by the Raspberry Pi Foundation and featured on their Blog (https://www.raspberrypi.org/flappy-brain/) and most recently was featured as The MagPi Magazine Issue 35 in their Amazing Pi Projects List (https://www.raspberrypi.org/magpi/)
The MagPi magazine is a a resource for everybody with a Raspberry Pi. It includes articles, project details and tutorials in everything from Scratch, to Python and C++ and has software and hardware projects.  Also, they announced recently that it is going to become a physical magazine soon and be available to buy in shops as well as the current PDF download and acces through the Apple App Store and Google Play stores.

It's been a fun few months and to me is an example of what can be done if you take advantage of the resources out there if you have idea.  For this project I used some great code and guide to hack the Mindflex.  Code from a 2008 Linux Format article and some previous work I did to figure out how to read serial data on the Raspberry Pi.

I will be bringing Flappy Brain to the next Egham Raspberry Jam. It is on the 12th of July 2015 and as always is free to attend.  All that's required is to register beforehand on the Eventbrite page.
http://eghamjam8.eventbrite.com

Maybe I'll see you there.

Friday, May 8, 2015

Created a game in Scratch

I came across some videos of Jeff Minter who created some amazingly original games in the 8 bit era and is still going strong developing new games today and it got me thinking of one of his stranger games that I remember playing on the Commodore VIC-20.
Meta Galactic Llama Battle at the Edge of Time.  Here's a video of it on the Commodore 64. I can't remember if I had it for the C64 as well, but definitely played it on the VIC-20


As with most of Jeff Minters games it kind of takes a slightly different approach.
You're a Llama at the bottom of the screen and you can move left and right.  You can exit one side of the screen and come in the other side as well.
You shoot lasers out of your mouth at 45 degrees and you have to shoot the spiders falling from the top of the screen.
If a spider reaches the ground it starts to crawl towards you making evasive action essential. 
All very interesting so far.  To add to the complexity and the control you can also raise and lower a ceiling for your laser to bounce off of to assist in shooting the spiders.

 I had great fun with this game all those years ago and so wanted to see if I could re-create something similar using Scratch.

After some fumbling I put created a little game inspired by the Llamas.

Below is the game and here is a Link to project on Scratch website if you want a peek inside. https://scratch.mit.edu/projects/61239650/




It's been fun developing a game in Scratch. I'm most definitely not an expert and I expect there are a lot of refinements that could be done to the code to improve the game but it plays similar to the original.  I hope you enjoy it.

Friday, May 1, 2015

7th Egham Raspberry Jam - 26th April 2015

What a great Sunday afternoon of Raspberry Pi activity.

40+ people turned up on the day with an ever greater proportion of parents with their kids with a real buzz on the day.

The most popular tables were the ones where a boy and girl were showing other kids how to program in Scratch.  I had a lovely experience at the end of the event where a father and his daughter came of to say they were leaving and the daughter was beaming as she had just written her first program in Scratch.  It was amazing to see the excitement on her face and the please she was experiencing from leaning a new skill.

For those interested in the latest in desktop environments +Martin Wimpress was there showing how fantastic Ubuntu Mate run on the Raspberry Pi 2 with desktop effects enabled. We had a great chat about RPi.GPIO support and what it would take to have it baked into the next build.
Martin also solved one of my questions. When navigating menus in Mate on the Pi2 sometimes they would stick and not respond.  I thought this was an I/O issue but it looks like it is down to OpenOffice have massive icons SVGs so on first opening a menu the Raspberry Pi has to render the icons for display. This happens once when you go to the menu for the first time. For return visits the icons are already renders so the menu is nice and fast.
I'm trying to decide if I will uninstall OpenOffice as Abiword and Gnumeric give me everything I need and then for graphics I use GIMP and Inkscape.
Martin had a build of Ubuntu Mate that used F2FS (http://en.wikipedia.org/wiki/F2FS) This is a file system specifically designed for flash memory and is suppose to improve the life of SD cards as well as improve performance.  I haven't had a chance to test yet but another win for optimisation.

As a side note GIMP and Inkscape are definitely usable on the Pi2.  I have previously used both on hardware of similar performance for commercial website development.  It's not going to match your i5/i7 with 16GB RAM and the latest graphics card, but it is definitely usable.

A second side note not from the Jam. I had my Ubuntu Mate Pi2 set up at home and needed to print.
It found my wifi network printer and installed all the correct drivers without me having to do anything special.  I love that Graphics acceleration just work, wifi (with the right dongle) just works and printing just works. There are the 3 points that I usually have concerns about on Linux and with the Raspberry Pi they just work.

Back to the Jam details.
Among the many items on display were:

Music playing Octopus (see flickr link at the bottom for pictures)
Portable games console,
Robot arm controlled by Scratch,
Crumble board (http://redfernelectronics.co.uk/crumble/) A great little interface board programmed using an block based interface like scratch.
Minecraft fun
Robot with robotic arm and webcam all controlled using PWM to vary speed from an Android table.
Sonic Pi coding +Sonic Pi
Scratch Development
CNC machine controlled by Raspberry Pi by +Stephen Cornes
EEG Brainwave reading using Mindflex, Arduino and Raspberry Pi
7 Segment of Pi board for development
Pitrol - build it yourself controller for the Raspberry Pi

At the end of the event I got the TwitterCam running and it did take pictures and tweeted them
The pictures can be see at the @EghamJam feed

Here is a link to some pictures from the event if you want to get a flavour for what happens at the Egham Jam.

https://www.flickr.com/photos/tictactoepony/sets/72157649918333124/

What a great day and thank you to everyone who came and brought their projects and enthusiasm to share.


We also had some Motorola LapDock troubleshooting going on as well.  These things turn your Raspberry Pi into a laptop


EEG Brainwave reader with Ubuntu Mate Visualisation



Looking forward to the next Egham Jam.