My second LibriVox recording is finished! The basic idea behind LibriVox is to read public domain texts and to put the recordings back into the public domain. More about that on librivox.org.

As with the last recording, my personal motivation is to practice my rusty English pronunciation. By reading and rereading the texts I believe my English will become more fluent and in the end also achieve better articulation.

The second recording is "Double Take" by Richard Wilson, a science fiction short story available from Project Gutenberg. I have also made a quick LaTeX typesetted version of the Gutenberg etext for more comfortable reading and with slight corrections: doubletake_wilson_text.pdf (369 KB).

Story Summary / Teaser

Pacing through a high-speed spy adventure, young Paul Asher finds himself going around in circles -- very peculiar circles indeed!

Audio Recording - Runtime: 26:29

MP3 encoded with standard lame VBR preset (obviously better/smaller than CBR 128 kbps)

doubletake_wilson_tb_vbr.mp3 (19.1 MB)

Published a small, but very useful console tool to update md5sum or shasum digest files. It will read existing md5sum.txt files and add new files to it without rereading the whole directory tree.

This makes digup very useful to update and verify incremental archives like chronological data storages or music collections, which are nowadays commonly stored and backuped on hard disks. Using a full file digest scan even slowly creeping bad blocks on old hard disks can be detected. By using a crontab entry, this check can be performed unattended and routinely.

For more information, the source code and binaries for various platforms see the digup web page.

After two weeks of hard work, my first LibriVox recording is finally finished! The basic idea behind LibriVox is to read public domain texts and to put the recordings back into the public domain. More about that on librivox.org.

My personal motivation behind recording, besides a lasting inclination towards fantasy and science fictions texts, is to practice my rusty English pronunciation. By reading and rereading the texts I believe my English will get more fluent and in the end also achieve better articulation.

The first recording is "The Last Evolution" by John W. Campbell, Jr. a science fiction short story available from Project Gutenberg. I have also made a quick LaTeX typesetted version of the Gutenberg etext for more comfortable reading and with slight corrections: lastevolution_campbell_text.pdf (153 KB).

Story Summary / Teaser

A future Earth population is unexpectedly attacked by powerful extraterrestrial Outsiders. However, the highly developed civilization of humans and machines quickly builds up forces and defends itself. This desperate, life threatening struggle against the invading aliens leads to The Last Evolution.

Audio Recording - Runtime: 58:42

MP3 encoded with standard lame VBR preset (obviously better than CBR)

lastevolution_campbell_tb_vbr.mp3 (42.7 MB)

If someone drops me a line with a good reason for uploading 250 MB, then I can also provide a lossless 24-bit FLAC file. Just burning a CD is not a good enough reason.

The recording is now also available on archive.org in the LibriVox Short Science Fiction Collection 026.

Released a minor updated source code version for Flex Bison C++ Example. The example source code is released into the public domain or, at your option, under the Do What The Fuck You Want To Public License (WTFPL).

This minor bugfix release fixes up two simple compilation issues with the newest bison version 2.4.1.

For more information and the download package see the Flex Bison C++ Example web page.

While publishing screenshots for CryptoTE (nice cross-platform text editor with integrated cryptography) the idea of making a short tutorial screencast came up. A screencast is basically a video made directly from the computer screen with no camera involved. Usually speech comments are added to create a sort of impromptu screen presentation. Thus I could show off the cool features of CryptoTE, and people could see what the program is about without installing it.

So far the cool and very trendy idea, now for sorting out the technical difficulties in making such a video. In theory a screencast video should be well compressible with a lossless codec. Unresized screenshots in full-resolution are also better compressed using the lossless PNG format than with JPEG, the PNG files are much smaller and show no blurry edges. Similarly screencasts, being only a sequence of full-resolution, losslessly recorded screenshots with some little mouse-movement, should be compressed losslessly.

On the digital inspiration blog I found a detailed "Screencasting Software Guide - Review of Desktop Screen Recording Tools" for all platforms. However, because my primary operating system is Linux (and since I also wanted to show that in the screencast), the range of tools that actually work is very limited.

Capture Tools: Tried xvidcap, vnc2swf. Picked vncrec!

The first tool that popped up while searching was xvidcap. It is not related to the XviD codec, but derives its name from X Window. It is a small tool that can capture any activity on the X Window desktop into a variety of different formats. Among these formats are MPEG, MPEG2 and many other compressed formats supported through ffmpeg.

However, after a few trials it became clear that xvidcap was too slow. Even though it captured the screen only with 10 fps (frames per second), the encoding process or capture process itself was not fast enough for a perfect recording of screen activity. I had large frame drop rates and moving the mouse cursor didn't look very good with only 10 fps. I presumed that with so much computer power in present-day CPUs, a higher quality had to be achievable somehow. Some other minor problems occurred with xvidcap: dialogs popping up in the wrong places, and context menus went outside of the capture area.

After some research I found a very good solution: VNC (Virtual Network Computing) recording. Using VNC one can basically connect to a remote desktop and control the mouse and keyboard via the network. Due to prior experience with TightVNC, it was my software of choice and I quickly set up a secondary user account on my desktop computer. Another advantage of VNC and the secondary user account was that I could easily configure the X desktop to look pretty for recording at the uncommon resolution 800x700 pixel, which was needed to show all dialogs without clipping.

For recording VNC sessions there are multiple solutions, of which vnc2swf was the most commonly mentioned for creating screencasts. However, I had only lots problems with this software; the output files were enormous, there was no easy support for adding simultaneous audio and the basic version didn't even have a seek bar to jump back and forth in the video. Nevertheless, the idea of encoding RFB (Remote Framebuffer, the protocol behind VNC) messages as Flash operations is very good.

Much better suited for my purpose was vncrec. It fires up a normal VNC client session and records all mouse, keyboard and X11 display events in a .vnc file. Thus it essentially only records changes to the displayed desktop, exactly what I needed, and even in lossless quality. The .vnc files can be replayed using the -play directive.

Encoding: Examined Lossless Codecs, Resorted to MEncoder and x264

Next step was to convert the .vnc recordings into video files that are playable using a Flash video player. My favorite Flash video player is Flowplayer and a comprehensive guide to installing it is available on their website. Since Flowplayer uses Flash to play the video, the available codecs are limited to those supported by Flash. I discovered that Flash has a special lossless codec for "screen videos", it is labeled flashsv in the ffmpeg formats list. However, a few tests showed it to be way larger than a well compressed H.264 video. Using other encoding tools than ffmpeg did not show any improvement.

So I basically gave up on creating a lossless compressed version and resorted back to x264, because it is the currently most advanced free codec and H.264 video is supported by Flash. Thus the .vnc must be compressed to a .mp4 file. For this purpose I used the mencoder tool, part of the MPlayer suite. Mencoder cannot read .vnc files directly. Apparently transcode, an alternative Linux video tool suite, can read vncrec files via a plugin, however, the resulting quality was not agreeable. There seemed to be some problems in the RGB to YUV2 color-space conversion of transcode which resulted blurred dialog box edges.

The final solution was then to use an enhanced fork of the vncrec tool by twibright labs. This version contains some YUV4MPEG2 export code which produces a good looking YUV2 raw stream. These streams can be directly piped into mencoder, without any intermediate files.

Vncrec has no support for recording audio simultaneously, so that must be done with an extra audio tool like Audacity or plain ALSA arecord. Using mencoder the resulting audio file can be encoded using FAAC and then multiplexed alongside the H.264 video stream.

Final Sequence of Commands

First launching a VNC server on the secondary user account. More work is needed to get a complete desktop up and running. The following few lines are only the basic steps to start the VNC server display on :1. SSH is used instead of su because otherwise lots of environment variables are kept from the primary user.

$ ssh <recording-user>@localhost
Password: <enter password>
$ vncserver :1 -geometry 800x700 -depth 24 -deferupdate 0

You will require a password to access your desktops.

Password: <some temporary password>
Verify: <same again>
Would you like to enter a view-only password (y/n)? n

New 'X' desktop is box:1

Starting applications specified in ~/.vnc/xstartup
Log file is ~/.vnc/box:1.log

$

Returning to the primary user account, one can connect to the VNC server using the recording client, or any other VNC client for that matter:

$ vncrec :1
VNC server supports protocol version 3.8 (viewer 3.3)
Password: <temporary password from above>
VNC authentication succeeded

After setting everything up for recording, connect using the VNC capture program.

$ vncrec -record screencast.vnc :1
VNC server supports protocol version 3.8 (viewer 3.3)
Password: <temporary password from above>
VNC authentication succeeded

Now to do the recording. All VNC actions are saved to screencast.vnc. Audio is not automatically recorded as well. To capture speech, plug in a microphone and record it simultaneously using an audio recording software. I found it convenient to first start audio and then VNC, and later synchronize both by deleting sound prior to the ENTER key following the VNC password entry.

To terminate the recording, press F8 and select "Quit Viewer" or just close the VNC window. This will correctly stop recording and flush the capture file.

After recording, the VNC server can be terminated. As the secondary user:

$ vncserver -kill :1
Killing Xvnc process ID 24148

The resulting VNC recording can be replayed like this:

$ vncrec -play screencast.vnc

Prior to encoding, the VNC recording can also be viewed directly by piping the raw YUV2MPEG into MPlayer, note the switch telling MPlayer the stream type:

$ VNCREC_MOVIE_FRAMERATE=25 vncrec -movie screencast.vnc | mplayer -demuxer y4m -

For stream editing it is most useful to create a raw file without delta frames. For this purpose the HuffYUV or FFmpeg HuffYUV codecs are best used. HuffYUV files can also be processed by Windows software.

$ VNCREC_MOVIE_FRAMERATE=25 vncrec -movie screencast.vnc | mencoder -ovc lavc -lavcopts vcodec=ffvhuff -o screencast-ffvhuff.avi -demuxer y4m -

Finally the video and speech audio file can be encoded using x264 and FAAC into a .avi file.

$ VNCREC_MOVIE_FRAMERATE=25 vncrec -movie screencast.vnc | mencoder -ovc x264 -x264encopts crf=23 -oac faac -faacopts quality=100 -o screencast-temp.avi -demuxer y4m - -audiofile screencast-speech.wav

For display on the web, the .avi file must then only be repackaged as a .mp4 Matroska container for display on the web using a Flash player.

$ MP4Box -aviraw video -out screencast screencast-temp.avi
$ MP4Box -aviraw audio -out screencast screencast-temp.avi
$ MP4Box -new -add screencast_video.h264 -add screencast_audio.aac -fps 25 screencast.mp4

Summary

With vncrec great screencasts can be made on Linux using a VNC server. However, the available lossless video compression techniques are not as far developed as lossy movie compression, so I resorted back to using the x264 codec. A special patched version of vncrec allows easy piping of the uncompressed picture stream into MPlayer. Using mencoder this raw stream can be converted into many different formats and simultaneous speech can be also added.

In the end most work when into scripting and recording a fluent screencast video, because the post-production editing facilities available are not very rich. The only tools available for that are Avidemux and Audacity. Maybe you will have more luck with Cinelerra than I did.

You can see the finished tutorial screencast about CryptoTE by following the image link below:

After almost one year of personally testing the program, I decided to publicly released the first version of CryptoTE v0.5.390. One year in the making, CryptoTE is a very useful little text-editor with integrated cryptography. The name stands for CRYPTOgraphy Text Editor and it transparently encrypts text files storing them into secure containers. The program incorporates the popular editing component Scintilla and makes heavy use of wxWidgets.

For more information, the source code and binaries for various platforms see the CryptoTE web page.

Today I gave the final completion presentation for my diploma thesis. The talk showcased a selection of results published in the thesis. Results and experiments are only sketched, as all further detailed information can be found in the thesis PDF itself.

The talk contains side-by-side comparison plots of feature enhancements made to ns-3 and verifications thereof using ns-2. Furthermore, the EDCA extensions implemented in ns-3 are tested against analytically calculated reference values. In the end, a speed test comparison is done between ns-2 and ns-3, which uses the implemented classes to run an experiment scenario identically on both simulators.

The slides are available as PDFs in following two variants:

Presentation Slides (including appendix):	ns-3-wifiex-completion-slides.pdf	1455 kB
Presentation Slides (two per page, excluding appendix):	ns-3-wifiex-completion-slides-1x2.pdf	1040 kB

Here the table of contents:

1. Thesis Objectives
2. Enhancements
   1. Propagation Loss Models
   2. Reception Criteria
   3. Frame Capture Effect
   4. EDCA Implementation
3. Speed Comparison
4. Conclusion
5. Appendix
   1. Enlarged Plots and Figures
      1. Propagation Loss Models
      2. Reception Criteria
      3. Frame Capture Effect
      4. EDCA Implementation

After a very exhausting last week with lots of writing and little sleep, my diploma thesis is finally complete. The thesis is on enhancements to the 802.11 model and EDCA (enhanced distributed coordination access) QoS (quality of service) extensions in the new network simulator ns-3.

The thesis is written in English and a copy of the abstract and table of contents is located below. There is also a Zusammenfassung in German.

It is available as PDF in different variants:

Standard PDF version:	thesis-bingmann-ns-3-wifi.pdf (3469 kB)
Better printable version (with black links):	in RGB color with binding offset (3482 kB)
	in CMYK color with binding offset (3381 kB)

Shortened Table of Contents

• 1. Introduction
     1. Motivation
     2. Contributions
• I   Related Work
  1. IEEE 802.11
     1. 802: The Big Picture
     2. Outline of 802 and 802.11 Layers
     3. PHY Layers
     4. MAC Layer
  2. The ns-2 Network Simulator
     1. Overview
     2. Problems with 802.11 and Overhaul by DSN
  3. The ns-3 Network Simulator
     1. Design Overview
     2. Architecture of 802.11 Implementation
• II   Enhancements
  1. Propagation Model Enhancements
     1. Propagation in ns-3
     2. Basic Propagation Loss Models
     3. Further Models in ns-3.4
     4. Nakagami-m Fast Fading
     5. Implementation and Verification
  2. PHY Layer Enhancements
     1. Modeling the Transceiver
     2. Implementation of Cumulative Noise
     3. SINR Reception Criterion
     4. Frame Capture Effect
     5. Implementation Issues
     6. BER/PER Reception Criterion
     7. Verification
     8. Discussion of Reception Criteria
  3. EDCA QoS Extensions
     1. Modeling DCF
     2. Extending Model with EDCA
     3. Implementation Issues
     4. Verification
  4. Speed Comparison – ns-2 vs. ns-3
     1. Highway Lanes Scenario
  5. Conclusion
     1. Summary
     2. Future Work
  6. Bibliography
  7. Background
     1. A Note on Decibel
  8. Extra Figures and Tables
     1. 802.11a Convolutional Encoder
     2. Default EDCA Parameters
  9. ns-3 Crash Course
     1. Callbacks
     2. Objects, Ptrs, Attributes and TraceSources
     3. Highway Lanes Scenario Code

For the last three months I have been working intensely on my diploma thesis. The thesis will be about 802.11 enhancements and EDCA QoS extensions in the new network simulator ns-3.

Today I gave my halftime presentation about the current status of my efforts. The talk is composed of a short introduction into ns-3, followed by a detailed discussion of WLAN packet reception criteria and finishes with a review of DCF and how EDCA extends it.

Be warned: many slides are not all self-explanatory and therefore less suitable as a casual introduction into the topics. They are slides meant for presentation.

The slides are available as PDFs in following three variants:

Presentation Slides (including "Animations"):	ns-3-wifiex-halftime-slides.pdf	1076 kB
Handout Slides (one per page):	ns-3-wifiex-halftime-handout.pdf	1036 kB
Handout Slides (four per page):	ns-3-wifiex-halftime-handout-2x2.pdf	1028 kB

Here the table of contents:

1. ns-3 Basics
   1. Introduction
   2. Showcase: Design Patterns
   3. Current State
2. Wifi in ns-3
   1. State of 802.11
   2. PHY Layer
   3. Signals, Noise and Interference
   4. Short Recapitulation of DCF
   5. QoS with EDCA
3. Conclusion

Released yet another updated source code package for Flex Bison C++ Example. The example source code is released into the public domain or, at your option, under the Do What The Fuck You Want To Public License (WTFPL).

This bugfix release solves a subtle, severe bug, which rendered the template code useless. Even the example exprtext program segfaulted with any expression.

Corrected a very subtle bug with the newly introduced virtual yywrap() function in the FlexLexer class. Depending on how the header was included, the class contained the virtual yywrap() function or not. These differing class declarations lead to very strange NULL pointer exceptions, because the different compiled objects assume different class memory layouts. Ultimately the exprtest program always segfaulted.

For more information and the download package see the Flex Bison C++ Example web page.