Extending HUMAX iCord – part 3

HUMAX iCord HD open source tools become mature

Long long time ago and in between things have changed a lot .. today is “T-Day”, the development team of iCordForum.com community has updated all tools for the HUMAX iCord HD. Find below the official press release (in German and English).

German press release

T-Day beim iCordForum.com
Update aller relevanten Tools für den Humax iCord HD

Das rein privat betriebene und größte deutschsprachige Forum zum Humax iCord HD veröffentlicht heute zu allen entwickelten Tools neue, deutlich verbesserte Versionen.

Der alternative Kanallisteneditor Omega bietet verschiedenste Möglichkeiten, um die Kanallisten des iCord HD komfortabel zu bearbeiten und zu erweitern. Die Kanallisten weiterer HD-Receiver, wie vom HD-Fox oder dem neuen HD-Fox IR, können mit diesem Editor ebenfalls optimiert werden.

Mit der Entwicklung von Platoon wurde erstmals die Möglichkeit geschaffen auch Videos (sowohl SD als auch HD) auf dem iCord wiederzugeben, die nicht mit einem iCord aufgenommen wurden. Somit kann der iCord-Benutzer nun auch für Urlaubsvideos, Aufnahmen vorheriger PVRs oder ähnlichem von der Wiedergabequalität des iCord HD profitieren.

MiniRootFS – Cortex ist die Weiterentwicklung des ersten verfügbaren Patches der Firmware. Dieser erweitert den iCord HD u.a. um die Funktionen eines Media- und Sambaservers. Die zusätzlichen Dienste lassen sich über ein Webinterface steuern. Von dort aus können auch die Aufnahmen auf dem jeweiligen Gerät bequem verwaltet werden und stehen über den Webbrowser im gesamten Heimnetzwerk zur Verfügung. Updates und neue Funktionen lassen sich direkt über das Webinterface von iCordForum.com laden und installieren.

Bei iCordForum.com handelt es sich um eine im Jahr 2007 gegründete, rein privat betriebene Initiative. Inzwischen hat sich um das Forum eine Community von über 14.000 iCord-Besitzern gebildet.

Das Entwicklerteam des Forums hat mit den am heutigen Tage aktualisierten Tools auch außerhalb des deutschsprachigen Raums für Aufsehen gesorgt. Aus diesem Grunde werden alle Entwicklungen multilingual (bisher deutsch und englisch) ausgelegt.

English translation of the press release

T-Day at iCordForum.com
Update of all major tools for the Humax iCord HD

iCordForum.com, the private maintained and largest German forum for the Humax iCord HD, is publishing today new and improved versions of all developed tools.

Omega, the alternative channel list editor, offers different capabilities for comfortable editing and extending of the iCord HD channel lists. The editor does work as well with other Humax HD settop boxes like HD-Fox or HDFox IR.

Platoon allows at the first time to view non iCord HD recorded video files on the iCord HD after a format conversion. It extends the iCord HD to a multimedia player for private videos and recorded video streams from other PVRs. It enables the user to utilize the high quality video playing capabilities of the iCord HD.

MiniRootFS – Cortex is an update of the first iCord HD firmware patch. This extension provides amongst other capabilities a Samba and a media server. All services can be controlled over a Web UI. This user interfaces also provides an access to the recorded video streams to view and organize these. The streams are available for viewing on all devices in a home network. The Web UI furthermore allows the maintenance and update of all provided components and services.

iCordForum.com has been founded in 2007 as a private initiative. In between the iCordForum.com community has more than 14,000 registered users.

The iCordForum.com development team could also gain a lot of attention outside Germany with the tools updated today. Therefore all tools will be provided with multi-language support (currently implementations are available with German and English language support).

You’ll find all tools for download at http://www.icordforum.com/index.php?page=Thread&threadID=6633. The whole stack has been published under GPL license, so far everyone can download and use; nevertheless for download you need to register at iCordForum.com.

MiniRootFS Cortex with Web UI for video streaming and system maintenance

As I’m part of the iCord HD firmware (root filesystem) extension dev team and responsible for the Web UI I’ll write some more in detail about this particular part. “MiniRootFS Cortex” is the name of the firmware extension, “MiniRootFS” stands for the extension and “Cortex” for the release (currently 1.1). What does MiniRootFS Cortex provide in a nutshell?

  • Extension of iCord’s original MIPS Linux (Busybox and uClibc) based operating system with new services like SSH Server, SFTP Server, Samba Server and Telnet Server
  • Package management system ipkg (see http://en.wikipedia.org/wiki/Ipkg) for further extensibility
  • Web UI for service maintenance, package installation and system configuration / update
  • Video Browser for browsing, viewing and organizing recorded video streams (works with VLC browser plugin)

One of the iCordForum.com members has created some youtube videos that show the different capabilities in detail ..

Overview about the iCord HD Web UI

iCord HD – Web UI overview

  • Overview about the Web UI
  • Demonstration of package installation
  • Video browser capabilities (shown here is the beta version with some missing functions like moving and deleting videos)
  • VLC player usage
Installation of VLC player and browser plugin

iCord HD – Installation of VLC

Access with WinSCP, FTP, PuTTY and other clients

iCord HD – WinSCP and FTP access

iCord HD – PuTTY access

iCord HD – Google Chrome

iCord HD – Windows network

iCord HD – Windows Media Center

iCord HD – (Almost) live TV

Package update with new video browser

iCord HD – Package update

  • Demonstration of package update
  • New capabilities of video browser (still beta with some layout issues, for release layout see the picture below)

Screenshot-HUMAX iCord HD - Mozilla Firefox

MiniRootFS Cortex installation

The installation of MiniRootFS Cortex works with following pre-requisites

  • HUMAX iCord HD firmware 1.00.17 or
  • HUMAX iCord HD firmware 1.00.18;
  • both with any pre-release or beta of iCordForum.com firmware patch

The steps are the following

  • Copy hdpvr.hmx and kernel17.tar to an USB stick
  • Plug in the USB stick into iCord HD
  • Switch off iCord HD and switch on with pressed “Record” key
  • The hdpvr.hmx will be recognized and written into iCord’s flash memory
  • After the reboot message switch off and on again, keep USB stick plugged in!
  • iCord will boot normally with TV on screen, the patch will be executed in background (app. 5 minutes) depending on the file kernel17.tar on the USB stick
  • After patch exexcution the original kernel will be rewritten automatically into iCord’s flash
  • The iCord will restart automatically after patch execution

You should now be able to reach the iCord Web UI with http://<ip-address>.

User: root
Pass: icordHD

The password is always the same for all services. Afterwards you should configure your network over the Web UI (Network), this switches off the original HUMAX iCord HD network configuration and gives you full control about the network properties for all purposes.

Support

It’s important to get help when struggling with some issues. The developers team and the iCordForum.com community provide offline support and in most cases we were able to solve all problems. That’s the advantage of community driven software development, so the easiest way to get hep (in German and English language) is to register, search and read and post questions.

Developers are welcome!

We’re still looking for developers that are willing to provide some time to help making iCord HD tools better. Easiest way to participate is to register at iCordForum.com and ask one of the developer leads how to join the team.

To set the hurdle as low as possible for a quick start I created 2 additional resources for newcomers. The first one is a sample package called hello. It contains an sample extension of the Web UI and can be downloaded and inspected at http://www.icordforum.com/index.php?page=Thread&threadID=6385. If you need some assistance feel free and ask the forum for further details.

Second piece is a virtual environment for testing purposes. It contains qemu (a virtual environment for MIPS (amongst others) running under Linux and Windows and a full configured virtual image running the base iCord HD firmware including MiniRootFS. It’s still pre-release (1.0.1) but the package structure is always the same so development could be started here. You can download the virtual environment at http://www.icordforum.com/index.php?page=Thread&threadID=6384.

What’s next?

Someone might ask for next projects that are in our minds. These are only ideas but to give you a clue what might happen and to motivate you to participate  I’ve collected some thoughts ..

  • Adapting Lua as scripting environment with a browser based Ajax driven UI for the Web UI
  • Remote programming of video stream recording
  • Browser based channel editor and system configuration

Disclaimer

It’s important again to refer ro the following statement .. The development team doesn’t take responsibility for any damage on your iCord HD. This is also included in the licenses by now. But anyway, we didn’t experience any case of damage of the iCord HD caused by our tools. You may also downgrade at any time to the original HUMAX firmware.

So far .. enjoy the new capabilities!

Extending HUMAX iCord – part 2

The main problem of the first shot was that there’s still included commercial software in the patch (BROADCOM driver). So far the next evolution was to create a patch for the iCord that doesn’t contain ANY commercial parts.

Some weeks ago I’ve become a member of the development team at http://www.icordforum.com. This is a non commercial community of people for answering questions around HUMAX settop boxes, especially iCord HD. The other objective is to develop valuable extensions for the iCord HD (without legal issues).

Yesterday we finished the first firmware patch for iCord HD that does NOT contain any protected code and components. The main capabilities of this patch are

  • Web UI for administration purposes
  • Telnet access
  • SSH access
  • FTP access for all directories
  • Samba access

Under the cover we’re using ipkg package management system to be able installing further extensions. All common functionality is still available of course. As I mentioned, the patch doesn’t contain any proprietary HUMAX od BROADCOM code.

The development team doesn’t take responsibility for any damage on your iCord.

You may download the patch under http://www.icordforum.com/index.php?page=DownloadDBData&dataID=236, prior you need to register. One hint .. the icordforum.com has an area for English speaking people too. The patch contains a readme.txt with installation instructions, I’ve translated these for non German speaking people.

  • Copy hdpvr.hmx and kernel17.tar to an USB stick
  • Plug in the USB stick into iCord HD
  • Switch off iCord HD and switch on with pressed “Record” key
  • The hdpvr.hmx will be recognized and written into iCord’s flash memory
  • After the reboot message switch off and on again, keep USB stick plugged in!
  • iCord will boot normally with TV on screen, the patch will be executed in background (app. 5 minutes) depending on the file kernel17.tar on the USB stick
  • After patch exexcution the original kernel will be rewritten automatically into iCord’s flash
  • The iCord will restart automatically after patch execution

You should now be able to reach the iCord Web UI with http://<ip-address>.

User: root
Pass: icordHD

The password is always the same for all services.

We added a feature to install TWONKYMEDIA server with an install button. As almost 80% have a fix IP address configured for their iCord it’s neccessary to add the gateway and DNS manually due to a minor bug in iCord’s network configuration. Open the file /etc/opt/network.conf (e.g. with FTP) and enter your network gateway and DNS server. Below you can find mine ..

[code]# Config file for setup of DNS and default Gateway (Router)
# DHCP user don’t need it!
# Please add IP Addresses of your Router and DNS Server.
# example:
# ROUTER=192.168.178.1
# DNS=192.168.178.1

ROUTER=192.168.2.1
DNS=192.168.2.1[/code]

Enjoy!

Please refer with all questions to icordforum.com, I’m a member there and will respond together with other people.

Extending HUMAX iCord – part 1

A case study for MIPS based embedded computing

This case study will describe some aspects of embedded computing and show how such systems with non-i386 architecture could be used for a extended stuff. All I’m describing here is for evaluation purposes and not ready for mass production but it may help to understand the hurdles and how to jump over when dealing with limited resources.

What should be reached finally?

I’ve bought a settop box for DVB-S2 some months ago, a brand new HUMAX iCord HD (http://www.humax-digital.de/products/iCordHD.asp). It’s an easy to use, reliable stuff with a fantastic high quality TV experience including broadcast recording. But over time I realized some missing capabilities like

  • a comprehensive channel editor on the system itself (without the need for export / import channel list),
  • remote timer list programming (I’d like telling my iCord to record an interesting broadcast even when I’m not @ home.),
  • a nice web interface for the stuff above and much, much more.

The hurdle is that the iCord is a closed box. The only way to talk with the outside world is through FTP; but FTP shows only the media directory and isn’t suitable for executing commands. But there’s a network connectivity and the capability to update iCord’s firmware with an USB drive containing hdpvr.hmx file (the firmware container). Unfortunately the firmware is encrypted but good news is that some resourceful people has found a way to decrypt.

One word to the legal aspect: Decrypting and providing a modified version of a vendor’s software isn’t allowed by law. So far you would find nor link to the decryption tool neither to the patched firmware at all. I’ll explain the how-to and provide some basic stuff, finally the root file system with some add-ons. If you’re looking for decrypting / encrypting iCord’s firmware you may find something in Google when looking for “colibri” in relation to iCord.

So let’s come to the first step ..

Extensions described in this part

After making all modifications described in part 1 you’ll have

  • SSH access to iCord with public key authentication (that’s being used because iCord’s BusyBox is compiled without login module)
  • SCP and SFTP support including SSHFS (mount iCord’s filesystem into your’s)
  • a bootstrap mechanism for further extensibility on /mnt/hd1 (the root file system is read-only but this is a way to add extensions later on without flashing again)

Experiencing the new behaviour

My nature is to try things first to get an impression what’s going on and afterwards digging into the details of the “how-to?”. So far you may start with downloading the ready squashed file system to insert into your firmware update file. Please read following instructions carefully before changing anything. And .. there’s no guarantee that everything is working, so be prepared to have the original firmware ready to avoid stress with your family about non-available TV ;-)

Instructions:

  1. Modify your firmware with the extended root file system Flash_700000.bin (it’s based on 1.00.17).
  2. Store the updated firmware on a Fat32 formatted USB stick.
  3. Generate a file “authorized_keys” on same USB stick; this file must contain the public key part of your rsa / dss keys generated with ssh-keygen or PuTTY. You can find a detailed description of SSH public key authentication and key generation under http://hkn.eecs.berkeley.edu/~dhsu/ssh_public_key_howto.html.
  4. Plug in the USB stick into your iCord, switch on (with the back side power toggle) and hold “Record” key on the front side panel during boot. You should see the download and update progress on the screen and also on iCord’s LED panel.
  5. When finished, you’ll get a message to reboot. Important! – let the USB stick plugged in. Switch the iCord off and on again. After reboot you can plug off the USB stick.

If everything was going well you should be able to access your iCord with

[code]ssh root@<ip-address>[/code]

You should be able to access the iCord now with SCP and SFTP, both can be used for any client you’d prefer to use. Under my Ubuntu installation on my laptop I also tried to get mounted iCord’s file system with

[code]sudo sshfs root@<ip-address>:/ /media/icord -o IdentityFile=/home/username/.ssh/id_rsa[/code]

When you’re browsing the iCord’s file system you will see 2 new directories, /root and /opt. Both are symbolic links to /mnt/hd1/root and /mnt/hd1/opt. Sense of /opt is to provide an anchor for further extensions without the need for flashing again. /opt/bin has been added to iCord’s $PATH variable (this directory isn’t there at the moment but could be created easily). /opt/etc/init.d may contain later on startup scripts that following the “S??*” pattern. The trick is that at system boot both directories, /etc/init.d and /opt/etc/init.d will be parsed; scripts contained in both will be executed in sorted sequence. Means if /opt/etc/init.d contains a script S50utelnetd this one will be executed between /etc/init.d/S40/networking and /etc/init.d/S55dropbear (the new one for dropbear SSH server). /opt/bin may contain than the appropriate binary utelnetd.

So finally you’ll have an iCord with full and secure access from outside world and the option to extend functionality without flashing again.

What’s behind – the “how-to?”

Input structure for modifications

hdpvr.hmx consists of 3 parts. The interesting one is the Flash_70000.bin (contains root_fs – root file system /). Once extracted from hdpvr.hmx, you’re able to copy the Flash_700000.bin to elsewhere for further work.

“Unsquashing” the root file system

Flash_700000.bin is a squash file system image. To extract the files I’ve used squashfs tools (http://packages.debian.org/search?keywords=squashfs-tools). Take care that you’re using version 3.1 instead of 3.3 due to an error that 3.3 produces when packaging the file system back to the image. I’ve no clue what’s the reason is but 3.1. worked for me well. The command to unpack the Flash_700000.bin is

[code]sudo unsquashfs -d /home/username/icord_root_fs/ /home/username/Flash_700000.bin[/code]

For next step let’s assume that you have a directory with unsquashed root file system on your Linux OS. All further links are relatively to this point. I’ve done all the work with Ubuntu 9.04, any other distribution should work either.

Adding dropbear SSH server and start script

The iCord is based on BusyBox (http://www.busybox.net/) and uClibc (http://www.uclibc.org/). A lean SSH server extension for BusyBox is dropbear (http://matt.ucc.asn.au/dropbear/dropbear.html). I took the sources of dropbear and compiled them statically linked for for MIPS / uClibc. The compiled binary you can found under /bin/dropbearmulti. There are also located symbolic links to this binary, dropbear, dropbearkey and scp. The start script for dropbear is located under /etc/init.d/S55dropbear. The host keys will be generated at system boot under /opt/etc/dropbear. Deleting these will trigger generation again. All client public keys are stored in /root/.ssh/authorized_keys file. You may change this manually or otherwise store it on an USB stick and plug the USB stick in during system boot. I’ve also modified the startup script /etc/profiles to avoid unnecessary error messages.

Adding greenend.org.uk sftp-server

Dropbear doesn’t support SFTP out-of-the-box. Therefore I’ve done some research to find a small sftp server under http://www.greenend.org.uk/rjk/sftpserver/. This isn’t ready for large production environments (and it will not be developed further) but I tested it and it works well for such an environment. The statically compiled binary sftp-server you can also find under /home/username/icord_root_fs/bin. As dropbear will look for a binary sftp-server under /usr/libexec I’ve modified the dropbearmulti binary with a hex editor and changed the sftp-server location to /bin.

Bootstrap mechanism

The extension point is quite simple .. adding a symbolic link /opt that points to /mnt/hd1/opt. All secrets you can find in /etc/init.d/rcS. It’s a simple script that contains the bootstrap mechanism. It mounts /mnt/hd1 first and looks for start scripts in /etc/init.d and /opt/etc/init.d in sorted sequence over both directories. Due to early mount of /mnt/hd1 I commented out the matching line in /etc/fstab.

“Squashing” the root filesystem

That’s it. The final step after all work is to create a squashfs image again. This could be done with

[code]sudo mksquashfs /home/username/icord_root_fs/ /home/username/Flash_700000.bin.new -be -all-root -noappend[/code]

You may check all further details in Flash_700000.bin (see link above). I’ll proceed with further testing and some more stuff continued ..

Credits

All this work wouldn’t have been possible with a some people who helped me with lot’s of answers and tools ..

  • colibri – programming of an outstanding tool to decrypt and encrypt iCord’s firmware
  • Anthony G. Basile – providing a qemu development environment for MIPS uClibc and answering a lot of questions (http://opensource.dyc.edu/aboutus)
  • Graham White – for some about runtime debugging (http://gibbalog.blogspot.com)
  • last but not least HUMAX – for a really high-quality DVB-S2 platform