Always-On Presentation

I made a device that mimics the sub-conscious of passive listening. The device records audio continuously. It then uploads the audio files to a remote location where those files remain in memory and are re-processed and combined to mimic the process of dreaming. Here the recordings are played back.

Prototype

I was able to get a working model in Processing using this code. I was able to leave the sketch running over night, recording in 1 hour intervals with no problems for 12 hours. This took up around 380MB at 44.1k 16 bit mono files. So I could get about a week’s worth of recordings with ~5.5GB.

auto-record-prototype-snap.png

One issue I had in the sketch was that using millis() was giving file sizes of varying degrees. I believe the problem was that in my IF statements and the fact that the operations would take longer than a millisecond, and therefore not get through the whole IF loop. So I jsut incremented a variable and had it reset to zero, giving me control over the interval in time as a function of the frameRate. I futzed around and got about an hour and twelve minute recording loop.

I took the files into Pro Tools when testing early on just to see how the transitions were, and it wasn’t all that bad, especially over a long interval. Here is a pic of the erratic bahvior of the millis() function set to record every 5 seconds. And a sample of that audio below.

always-on-pro-tools-snap.png

Project Update

Here is a drawing/sketch of a fully functional “always-on” recorder. For now, I am just shooting to get the stand-alone portion working and recording locally (no cloud transfer) before I leave for Spring Break in two weeks.

IMG_1016.JPG

With Danny’s help, I was able to get a sketch running in Processing that begins recording immediately. It saves the files as a pre-determined length with a manageable amount of dropped samples for the purposes of this device. I would like to make some way of having them overlap, but what could cause troubles when they’re being recombined. Danny and I also talked about a way to make a “mark” on each file so that you would know where and what to look for, like a button that appends a “check me” string onto the end of the file name. I could also knock the machine to make a loud sound which would be visible in an audio editor or could open a gate once the audio is analyzed.

On Sunday, I picked up a Raspberry Pi 3 Model B V1.2 from the ER, but once I got it home, I wasn’t able to power it on. I checked that my PSU was strong enough and reformatted the SD card and reloaded the OS, but nothing worked. I came back to ITP, got another Pi, this time it turned on, so I took it home and again, nothing. It would power on, but would not boot.

So today I came back and got a 3rd Pi, and Roland in the shop was able to help me get the OS up and running. First we tested that the Pi worked as another student (Mark from class) had a working Pi OS on an SD card. So once this new Pi booted up successfully, it was clear that my SD card was the problem. Roland directed me to go download Etcher which would reformat my SD card properly. I also downloaded the Stretch OS for Pi this time and not the NOOBS OS. I was able to load the OS ZIP file into Etcher and it formatted my SD card in a few minutes and… Pi booted up! I wish the Raspberry Pi website knew to direct folks to Etcher! I did not have time at school to test the Pi, but hopefully, during the next class, I can get the Processing sketch onto this Pi.

Research for Recorder

I checked out a Raspberry Pi B 3+ this week and tried to familiarize myself with it, but ended up not even being able to get OS loaded on to an SD card. There is a lengthy process where the SD card needs to be formatted in just such a way and it takes like 20-30 minutes each time and can fail right at the end :( I think that will be a viable option though at the B3+ has Wifi access, if I can get it to stream audio. Ideally the device would be able to save files straight to Google Drive.

I also investigated using iSpy which is security camera software, configurable for the home or office. It has the ability to take what appears to be web socket inputs and save those to the cloud. Here is a demo of that software (on my desktop Windows 7 machine at home).

Here is a list of web sites that I pulled up and saved in my research:

  • Getting Started with Raspberry Pi:
    https://www.pubnub.com/blog/2015-05-27-internet-of-things-101-getting-started-w-raspberry-pi/

  • Recording Audio on a Raspberry Pi:
    https://medium.com/@PTcampbell/recording-audio-on-a-raspberry-pi-3a51ae351b98

  • A USB Mic that could work with the Pi (I ordered one):
    https://www.amazon.com/gp/product/B00IR8R7WQ/ref=oh_aui_detailpage_o00_s00?ie=UTF8&psc=1

  • Record on Boot - Raspberry Pi:
    https://www.raspberrypi.org/forums/viewtopic.php?t=61741

Then I got on a kick thinking I could record straight to Google Drive via the command line on the Pi, so that took me down a rabbit hole. I’d have to make a dev acct with Google to get a secure key so that I didn’t have to log in to Google Drive each time, which also means that to use my NYU account, I’d need special keys. Here’s some links on that:

  • How To Upload To Google Drive from the Command Line:
    https://olivermarshall.net/how-to-upload-a-file-to-google-drive-from-the-command-line/

  • Github repo on Google Drive CRL Client:
    https://github.com/prasmussen/gdrive

  • Setting up the proper authorization account for Google Drive (which i did):
    https://developers.google.com/identity/protocols/OAuth2ServiceAccount

  • And the Google Dev dashboard for my NYU acct:
    https://console.developers.google.com/iam-admin/serviceaccounts?pli=1&project=always-on-recorder&supportedpurview=project

  • Google Cloud Platform (NYU)
    https://console.cloud.google.com/navigation-error;errorUrl=%2Fhome%2Fdashboard%3Fauthuser

I also emailed back and forth with Tom Igoe on the matter. He pointed me in a few directions, like Chapter 10 of his book “Making Things Talk” as well as PhoneGap, as he suggested using a mobile phone to be the recorder. I though perhaps an old iPhone or iPod, since using one’s current mobile device would be tempting to be distracting in the moment of inspiration.