Internet Explorer 9 (and Chrome sometimes as well) tries to be a little too smart and do too many things for you. The best example is removing the Flash movie from the DOM when you set its style to:
display: none !important;.
This small gotcha led to some headaches about why JS wasn’t able to call methods (via the ExternalInterface) on my Flash movie. As it turned out this way due to IE9 just completely removing the Flash movie from the DOM.
After having spent a good part of an hour on trying to install the Flash Player Debugger for FireFox on Ubuntu 11 64-bit, I found the following steps worked:
Thanks to: AskUbuntu.com.
Don’t forget to set up mm.cfg.
When all is done, run:
tail -f ~/.macromedia/Flash_Player/Logs/flashlog.txt
Recording the microphone using AS3 is quite easy. Actually, storing the data or compressing it and sending it to a server is where it gets complicated. Normally, a Google search helps a lot when trying to find good information, but on this particular subject there seems to be a lot of misinformation. There are also a lot of libraries and / or frameworks available, but so far they didn’t really do what I wanted to do or did it in a far too complicated way. I’ve provided a list of audio libraries I tried out at the bottom.
Clearing things up
When you record data from the microphone using SampleDataEvents, the actual data is (part of a) 32-bit floating point (Big Endian) 44.1 kHz mono PCM audio stream (assuming that you left the rate of the Microphone at its default of 44). To store the microphone input, all you need to do is append this data to a buffer.
To play back the microphone input you can use the Sound class. The Sound class can only deal with a 32-bit floating point 44.1 kHz stereo PCM audio stream as input. Lucky for us, we’re almost there, we just need to convert mono into stereo. As you can see in the example below, creating (fake) stereo out of a mono signal is a matter of writing each sample twice.
The documentation for the Sound class shows an example of how to merge several Sounds into one using the extract method. We can apply the same technique to play back our ByteArray using the Sound class:
The above seems simple enough, but when you want to save the data to a file, things get complicated. First you need to choose a file format in which to store the output. Secondly, if you want to reduce the amount of data to be saved, you need to choose a codec to compress the audio stream. When I was experimenting with MP3 encoding of my recorded data, I found out that the codec required the stream input to be a 16-bit signed integer with a sampling rate of 44.1 kHz. When you get into this territory, it helps to read up a bit about the techniques behind digital audio. I’ll try to summarize some of the most useful things I learned.
Pulse-code modulation (PCM)
Pulse-code modulation is a way of representing analog audio signal on a digital system by sampling the signal at a fixed interval. See 1) in the image I created to explain some PCM examples.
By keeping the same playback frequency, but adding more samples (using some form of interpolation), you slow down playback / lower the pitch (blue curve). The opposite is also true, remove samples and playback will speed up / the pitch is higher (red curve). See 2) in the PCM examples image.
If you multiply or divide the each sample by a fixed value, you increase or decrease the signal strength (green curve). See 3) in the PCM examples image. Keep in mind that multiplying or dividing could result in clipping or complete cancellation of the signal if you overflow or underflow the bit depth precision.
Sample rate conversion
The sample rate defines how many samples are taken of the analogue sound per second and is expressed in Hertz, for example, CDs have a sample rate of 44.1 kHz. The higher the sample rate, the better the quality. Other typical sample rates are 8, 11.025, 22.05 and 48 kHz.
There are two types of sample rate conversion, there is upsampling and there is downsampling.
It is usually easier (whether you are upsampling or downsampling) to first upsample (using linear interpolation) to the least common multiple (LCM) and then downsample to the required sample rate.
Generally speaking when you downsample an audio signal you can get away with duplicating a sample when upsampling to the LCM, most of the data will be dropped anyway as you downsample to the required sample rate.
Since sample rate conversion is quite CPU intensive, there are some shortcuts you can take to convert to a certain sample rate from another. Say you want to convert 44.1 kHz into 8 kHz, it seems a bit redundant to first upsample to 3528000 Hz (which is the LCM), and then downsample to 8 kHz. What you could do instead, is alternate between using every 5th and 6th sample (since 44.1 / 8 is roughly 5.5).
Bit depth conversion
The bit depth determines granularity of a single sample, the higher the bit depth, the more accurate the sound can be represented. CDs have a bit depth of 16-bit, other typical bit depths are 8, 24 and 32.
Usually 8-bit audio streams are using unsigned integers, 16-bit ones are using signed integers and 32-bit streams use signed floating point. Knowing this it is quite straightforward to convert from one bit depth to the other, it’s a matter of multiplication.
Note: You may need to do some additional checks to make sure you aren’t overflowing the available bits when doing the conversion.
When you understand the theory I just talked about, it becomes a lot easier to write the tools you need to convert audio from one bit rate to the other as well as changing between sample rates. It also gave me some ideas about some filters to implement, like a normalization filter. To help you get started I’ve included some tips and references below.
Tips
Further reading
AS3 audio libraries
Installing Adobe Alchemy on Windows7 wasn’t as straightforward as the Adobe Labs site made me believe.
The following error would keep on showing when I tried to run the example:
$ gcc stringecho.c -O3 -Wall -swc -o stringecho.swc
llvm-gcc.exe: error while loading shared libraries: ? …
I finally got it to work by using the steps below:
Note: If you already have Cygwin installed, you can install additional packages by running setup.exe again.
While writing a quick test application to record microphone input and playing that same recording, I found out that playing back a ByteArray directly using the Sound class wasn’t possible. Luckily a solution is never far away when doing a Google search and it turns out that FlexibleFactory has a good solution: the MP3FileReferenceLoader lib. With a bit of tweaking you can pass in a ByteArray directly and don’t need to rely on a FileReference.
Good news though! As per Flash Player 11, the following 2 methods have been added to the (native) Sound class that can be used to achieve the same behaviour:
I’m building an AIR app and am using Java to offload some heavy duty work to keep Flash responsive. I want to make sure the end user has a version of Java installed the is compatible with the Jar I ship with my AIR app. To do so I use the following (pseudo) code below:
To my surprise the result of the call to “java -fullversion” is send to the standard error stream and thus triggering the ProgressEvent.STANDARD_ERROR_DATA to be fired.
Small wtf moment today. I forgot to instantiate / give a String variable a default value. After appending the string and tracing it, “null” was the first part that showed, see below:
Fix:
Oops! 🙂
To properly close a transfer to the FTP server, you need to send an Abort command (see http://www.w3.org/Protocols/rfc959/4_FileTransfer.html (ABORT) ). Flex FTP doesn’t support this so I added some functionality to be able to send an abort.
First I added the Abort command to the Commands class:
After that I added a new invoker to the invokers package, AbortInv.
The FTPInvoker class required a new public method to support the Abort command, which can be overridden by the specific implementations. In FTPInvoker I added an abort method like this:
There was only one invoker where I had to override the abort method to get things working correctly and that was the UploadInv class. Here I added the following code:
After that it was a matter of updating the FTPClient class with an abort method to get the circle complete.
Unfortunately I also had to update handleData method in the FTPClient class a little bit, because aborting an upload would result into 2 (or more) resonses at once from the FTP server.
My handleData method now looks like this:
Links
Lately I’ve been working on a small upload tool in Flex/AIR.
To verify that an upload has succeeded, I created a hash (md5) of the entire file and sent that to the backend for verification. The AS3CoreLib contains a nice MD5 class, MD5Stream, but it’s much too slow for creating a checksum of files.
Actually, it’s so slow that when I was uploading a 5+ MB file the application would slow down so much that the upload speed dropped to a third of the speed without feeding the MD5Stream data during uploading.
Luckily, with AIR 2.0 NativeProcess has been introduced, it made it much easier to “outsource” complicated calculation to a process that can deal with that much better, i.e. Java.
The nice thing about Java is that it’s cross-platform just as AIR, and these days many people will have the JRE installed. So first of all, I wrote a small Java class that would print out an MD5 hash of a file based on the path that I passed. I jarred the class and added it to my AIR project.
The second step was to call my Java class instead of the AS3 MD5Stream class. This is surprisingly easy. The NativeProcess documentation comes with a pretty clear example and Piotr Walczyszyn has created a small framework, Flerry, to let Flex talk to Java and vice versa.
Looking at how Flerry is built will give you some more insight on how to deal with NativeProcess.
For my application, I only use the BaseStartupInfoProvider class from Flerry to find where Java is installed on the user’s machine. After that I set up all event listeners as per the NativeProcess documentation and call NativeProcess.start() with a NativeProcessStartupInfo as passed in parameter.
Now my uploads are fast again and I can generate a checksum in a fraction of the time I could before.
There are unfortunately some downsides to this approach. You have to deploy your AIR application as native installer to be able to use the NativeProcess class. So instead of having one AIR file, you’ll end up with four native installers (.exe, .dmg, .deb, .rpm). On the plus side, you can create the native installers from an AIR file without needing to resign the application.
Another downside is that now I’m relying on two runtimes being installed on the end user’s machine, AIR and Java.
My final thought is that now that I am outsourcing some calculations to Java, why not built the whole thing in Java? Then again, building a decent UI can take ages using Java. To me, my application feels like I’m creating a bit of a Frankenstein.
All in all I’m glad I now have a fast way to generate a checksum and my uploads aren’t slowed down by it any more.
Links:
Still working with Flex FTP and this time I was missing EPSV support. EPSV is roughly the same as PASV, exect that you only get a port number to connect to back and not an IP address. You can read up on RFC 2428 “FTP Extensions for IPv6 and NATs” if you want more details.
The first thing I did to add support for EPSV in Flex FTP is adding a response to the Responses class.
Second was the commands class, where I added the following line:
Then I added an extra argument to the FTPClient class’ constructor that will indicate whether or not to use ESPV.
I store the value internally and use a getter method to determine if an Invoker class (i.e ListInv) needs to use EPSV.
Based on the client epsv setting any invoker can be updated to send the EPSV command instead of the PASV command when needed, i.e. my implementation of ListInv’s startSequence method:
I also update the responseHandler method in ListInv to include the following case:
On top of that I made some tweaks to the PassiveSocketInfo class.
The parseFromResponse method now looks like this:
The createPassiveSocket method now takes two extra optional arguments (usingEpsv:Boolean and host:String) and just passes those on to the parseFromResponse method.
Hope this helps anybody looking for EPSV support for Flex FTP.
PS: It’s been a while since I added EPSV support, so let me know about any gaps.