A short description of the VIP1710s IPC algorithm Hans-Werner Hilse , 2010 Version: 2010-09-21 This document may be used by whomever for whatever she/he wants, given that she/he knows what she/he is doing and this licence is kept in effect on derived versions. (the know-what-you're-doing licence) This document should be downloadable at http://hilses.de/vip1710/libvipipcstack/README-IPC.txt in its most current version. Product names in this document belong to their respective authors, I do not claim them but use them to name the corresponding products. TOC: 1. Preface 2. Communication Infrastructure: Unix Domain Sockets 3. Communication Agents 4. Language: Message (De-)Construction 5. Misc 1. Preface I'm still playing with a piece of hardware that an electronics retailer here in Germany offered for 5 EUR, giving no promises for whatever usage scenario. It's a small little IPTV set top box (STB), able to decode MPEG1 and MPEG2 video (maybe more, it also contains a DSP that should allow for more compression schemes) and MP2 audio. The Box is based on the ATI Xilleon 210 SoC, which operates as the primary CPU on these Boxes (with a MIPS 4kec synthesized core operating at 300MHz). Unfortunately, specs for the chipset are somewhat closed. The WWW offers some few instances of documentation pieces that leaked, but they are mostly for other Xilleon based designs and even in sum they do not give a very good idea on how to drive the platform with your own software. In case of the STB the author of this document bought, a VIP1710, designed by Kreatel, later bought by Motorola, a closed-source userland software stack drives the functionality of the Xilleon platform. It consists of multiple software components, which are introduced in section 3 below. These software components communicate with each other and offer their functionality to the other components. An easy way to make some use of the platforms functionality, especially regarding its media streaming features, seemed to hook into this software stack, at least, until some major parts of it can be re-engineered. So the author started to examine what's going on between the software components. This document is the intellectual result of that work and is accompanied by some software: There's a unix domain socket traffic dumper which was named "sockdump" (find it at http://hilses.de/sockdump/) and finally a C implementation of the communication mechanism, "libvipipcstack" (you can find it here: http://hilses.de/vip1710/libvipipcstack/). The communication mechanism will be further referred to as "IPC", the well-known acronym for Inter-Process-Communication. 2. Communication Infrastructure: Unix Domain Sockets When looking at the software as it runs, running "lsof" and "strace", it was very easily determined, that the communication is based on passing messages on sockets. It uses Unix Domain Sockets, which share the socket API e.g. with IP sockets, which are far more popular. Unix Domain Sockets operate on a machine-level, though. There are processes that operate as servers, allowing one or more connections from connecting clients. You may be tempted to compare them to Named Pipes, but that would leave out the more general features of Unix Domain Sockets. In comparison to (named) pipes, Unix Domain Sockets have a broader idea of connection management. Plus, there's better buffer management, multiple connections on the same socket name, better connection state management etc. The vendor software stack uses an IPC mechanism that uses sockets that are being created in /tmp and generally have names with a "ipc_" prefix and a random-like appendix. Within the IPC protocol, these sockets are sometimes multiplexed and are identified internally with a suffix "#" plus a number, starting at 0. This suffix is not taken into action when creating, listening on or connecting to the sockets. So in the filesystem, the sockets look like this: /tmp/ipc_A2b4C8 /tmp/ipc_d9E75f while they are referred within the IPC communication as /tmp/ipc_A2b4C8#0 /tmp/ipc_A2b4C8#3 /tmp/ipc_d9E75f#0 In this example case, the first socket is multiplexed within the IPC stack to two "virtual" sockets. Connections to the same "physical" socket are assigned to the "virtual" sockets by looking in the IPC messages: they carry an address for the "virtual" socket they are meant to get sent to. More on this in section 4 below. 3. Communication Agents I'll give a short overview on some parts of the userland software stack here to give an idea of the software parts and their functionality: procman: This is the first process that gets started after system initialization. It does the following: * manage _services_. _services_, of which other software components do provide one or more, are started by and/or registered at the procman process. It offers the NameService, which is an information broker used to register and query service<->socket name associations. * the ProcessService allows to have procman start and supervise other processes. All processes (services) started/supervised by procman are run with individual uids and a given fixed gid. procman itself runs as the root user. * the TimeService seems to simply give access to clock data halserver: Gets run by procman before doing anything else. It provides some services that manage access to system resources. It runs as the root user. halserver manages, among a lot of other things, the platform watchdog timer and offers an IPC interface for "feeding" the watchdog. sysman: Operates close to halserver and procman. It manages the software stack but uses procman to start and supervise the according processes. sysman is also in charge of feeding halserver's watchdog interface. softwareservice: Seems to manage firmware as a whole entity. Not yet analyzed very closely, but it asks halserver for the crypto keys for the firmware. appman: Manages applications, which can be started on demand or via URI or MIME types. Other services register to this in order to get notified as applications start up or in order to request applications to get started. It also does supervision of application processes by sending them pings and waiting for replies. It uses procman services to actually start or kill programs. infobroker: Is a service, providing the InformationService which manages key-value associations, i.e. provides a common database. It is used for permanent storage (persistent configuration) and stores that in /flash/settings2.xml. It also maintains memory-only associations (volatile) which are also being used for pro-active status signalling for other applications (e.g. the current network address is stored there). It also offers a notification interface that triggers IPC calls upon changes to certain keyed data. netconfig: Well, even the name makes obvious what it does. displaymanager: Not yet very carefully analyzed, but it might just be a big, albeit not very mighty, X window manager. It queries for window sizes and it seems that's about it. Seems to trigger if a mouse pointer is visible, too... X: If you don't know on your own, you're not allowed to read this. mediamanager: Seems to manage stream sources. It works closely together with the streamer process. It gets some important resources from halserver (which makes them unavailable for other clients), like VideoMixer, VideoOutput and AudioMixer. It then offers its own services on top of these which are (normally) accessed by streamer and tv. streamer: This is kind of a media player backend. It has support for transports and container formats. It uses halserver's TsParser resource for doing MPEG stream parsing (as it seems, not just TS but also PS and PES) with the platform's hardware features. The streamer process supports different transport schemes which are differentiated by URI schemes: * HTTP: http://[:port][/] * RTPS: rtps://[:port][/] * IGMP: igmp://:port * UDP: udp://: * RTP: rtp://: * TCP: tcp://: * FILE: file:///path * LSTRM: lstrm:///path (Unix Domain Sockets) * LDGRAM: ldgram:///path (Unix Domain Sockets) tv: This is the media player frontend. It needs X+GTK for doing the status displays. (and a bit more I'm currently too lazy to describe right now. Check back in a few weeks if you cannot manage to find out on your own.) 4. Language: Message (De-)Construction So, that was a lengthy discussion of what else is there, but it was promised to describe the IPC mechanism, right? So it's time to see how the actual messages are formatted. It's not very complicated, so this section is a short one :-) #1: messages are stacks: Consider every message as a stack of data. #2: datatypes: there are Bytes (which are also being used to store booleans), Ints and Strings as far as I could see. Byte: well, not much to say about this. Int: 32bit, lower endian when you look at the message. Big endian if you're reading the messages backwards because I told you they're stacks :-) String: Can contain \0, so they are suffixed (i.e. "prefixed in stack order") with an Int value (as described above) indicating their length. They do not have to end with \0, but in practice, most of them do. #3: message header: all messages start with an Int giving the overall message length. #4: procedure calls: a lot of methods are procedure calls. These end (i.e. have at the top of their stack) in (reading from end to start) - a byte which is usually 0 (could not yet make sense of the exceptions to that rule...) - a string (encoded as described above) giving the procedure name - a string (...) giving the name of the called interface - a string (...) giving the name of the virtual socket name (see section 2) #5: replies: do not have the procedure call signature described above, but usually and on a single byte value indicating if there was an error (1) or not (0). Structure overview: RPC: Reply: If there are errors, they are commonly encoded (i.e. the reply looks) like this: When reading messages from a network socket, you can start by reading an Int - the message size -, create a buffer of that size and then read message size minus sizeof(Int)=4 further bytes. Received messages should be processed by pop'ing from the stack. 5. Misc This has been just a very small description on what's going on. If you're interested in the interfaces and how to use them, you should look at the libvipipcstack mentioned in section 1. It has prototypes for a lot of identified Interface/Procedure combinations and brings examples on how to use them in your own software. This is an ongoing approach and if you're interested in this, reading C header files should really be an ability of yours. That said, look int vipipc_prototypes.h in said library to see what already got identified. Use the software description above in section 3 in order to make sense of how the components interact. And then there's always: Trying, seeing what happens. No guarantees whatsoever are taken for the software and the information provided here. It may all be wrong, we may all live in different realities. I don't care since it works for me and you did not pay me, after all.