QNX is a multitasking, multiuser, POSIX-compliant, realtime, message-passing, microkernel, networking operating system. Multitasking means it can perform more than one task at the same time, allowing programmers to break up complex problems into separate sub-problems without having to worry too much about how the separate tasks interact. Multiuser means that QNX supports individual user accounts and provides protections and privileges to safeguard individuals' data.
POSIX-compliant means QNX adheres to a developing standard for operating systems interfaces which defines a basic set of commands which are to be available and some of the basics of how file systems work. In short, QNX looks like UNIX(tm).
Realtime means QNX was designed from the beginning to provide the response times required for realtime applications such as data acquisition or process control. Standard UNIX by design cannot guarantee any specific response time, be it milliseconds or seconds, making it inappropriate for demanding applications. In data acquisition, data collected at the wrong time is very likely the wrong data. In process control, inability to respond to critical stimuli in a timely manner can be disasterous. This isn't to say that you can't do data acquisition with standard UNIX; many people do, relying on extremely high-speed computers to deliver "virtual realtime" performance. We decided to go for the real thing which can provide realtime performance on machines as lowly as the 80286 at 12 MHz.
Message-passing means that communication between processes in QNX is performed by passing messages back and forth. The primitive commands required to send and receive messages are extremely simple, yet give tremendous power to developers for coordinating all the tasks required for their applications.
Microkernel is an approach to operating system design whereby the "kernel," or the heart of the operating system, is limited to supporting only those features which are absolutely required to be in the kernel. This includes process scheduling and interprocess communications (IPC). All ancillary operating system functions are provided as separate tasks which communicate with the microkernel and each other via messages. The Filesystem, disk drivers included, is a separate program as are the network interfaces. The advantage is that the user can tailor the components of the operating system to support exactly the applications s/he needs, allowing the use of less memory in simple monitoring systems, but supporting large systems as necessary.
Networking means more than the fact that you can network QNX machines together. Networking is virtually transparent due to the message-passing primitives. Passing a message to a process on another node is essentially the same as passing a message to a process on the same node. Therefore, once you have broken down your problem into separate processes, the processes can often be distributed to separate computers to take better advantage of available resources.
There is also a lot more to QNX. With POSIX-compliance comes the ability to port many UNIX applications. Support for TCP/IP makes remote support and communications possible. And the list goes on...
We have expended a tremendous amount of energy here at the ARP developing an architecture for Data Acquisition and Analysis which addresses these needs. Perhaps my primary goal in this development (after that of making the damn measurements) has been to simplify the specification of an experiment, ideally to the point where it can be read and understood by a normal human or even a scientist. An overview of the specifications is given in Development below, but first I'll outline some of the nuts and bolts involved in handling data under QNX.
The Telemetry Standards specify a rectangular frame, known as the "major frame," in which the rows are known as "minor frames." Each minor frame in turn includes synch words and possible frame counter data as well as the experiment data itself. The standard does not specify a word size, and some systems will break up data on arbitrary bit boundaries, but we have standardized on 8-bit bytes as the smallest unit of data. The data is transmitted one minor frame at a time at a predetermined rate.
Where and how often a particular datum appears in the major frame determines how often the datum is reported. For example, suppose a particular major frame contains 16 minor frames and the entire major frame is transmitted once per second. If you wanted to collect a particular pressure, Amb_P, 16 times a second, you would have to place that datum on every minor frame. If on the other hand you only wanted to collect it at 4 Hz, you would place it on every fourth minor frame. The data are collected immediately before each minor frame is transmitted, so the data which appear less often are collected less often. Distributing the data like this throughout the frame helps to even the load for collection and transmission, although it has the undesirable side effect of causing low-rate channels to be collected at odd times.
Previous incarnations of ARP Data Acquisition actually required the experimenter (or more likely the programmers) to position each datum in the telemetry frame. One of the biggest achievements of the current development effort has been the elimination of this chore. Given definitions for the required data, an appropriate frame is generated automatically. This greatly reduces the work required to modify an existing experiment.
The data clients locate the DG and receive their data via messages. High-priority processes are arranged in a ring: the DG sends data to the first client in the ring who processes the data and forwards it to the next and so on. Client applications may include screen-display, data logging, telemetry output, data-dependent algorithms, data extraction and data buffering.
Some clients may require considerable CPU or other resources to process the data they receive. This makes them undesirable as ring members, since their heavy processing might slow down the data transfer on the ring and even cause the DG to die due to data backup. These clients are best run not as ring clients but as buffer clients. The data buffer supports a somewhat different protocol for handing out data and is much more forgiving of slower processes. Most if not all clients can be run either as ring clients or buffer clients.
The separation of this command function into client and server has two benefits. First, the client program does not need to run on the data acquisition computer, but can run more appropriately on the data display computer (when the two are distinct, that is). This makes it easier to start up and shut down a display station without undue interference on the data acquisition in progress. Second, more than one client can run on more than one node. Since the server is just reading messages, it doesn't care if the messages come from several different clients.
Perhaps the pinnacle of experiment control, algorithms are a special combination of command client and data client. Algorithms see the data as data clients and send commands as command clients. The overriding architectural guideline is:
Under autonomous operation, every command decision should be based exclusively on information recorded in the data stream.The reason for this is that it guarantees that the reasons for a command decision can be determined after-the-fact by re-running the algorithm with logged data. This could be critical during field efforts. If an error in coding an algorithm produced an incorrect command sequence, you must be able to determine the cause to assure that the same error won't arise again. If a decision is based on the contents of some file or a value read directly from the hardware, this cannot be reproduced reliably during debug.
That said, algorithms provide tremendous power for automating experiment control. Decisions can be based on any data reported in the data stream. Watchdogs can be defined to trigger an action or a message in the event that a channel or channels strays outside their proscribed limits. Separate "threads" can be established so watchdog functions can operate independently of the main algorithm. (i.e. the watchdog may try to correct an over-temperature condition without interrupting the main algorithm before mandating a complete shutdown).
Since the algorithms are command clients, all commands issued by the algorithm are logged by the command server, giving an English summary of the operation of the instrument.
cmdgen is the command compiler. It processes a restricted context-free grammar and produces C source code for both client and server applications. As with tmc, it is easy to insert C code into cmdgen programs, giving developers considerable power and flexibility.
tmcalgo is the compiler for algorithms. It gains its power not only from C, but from both tmc and cmdgen. The instrument commands are specified in English, as specified in the cmdgen program, and the output is tmc code, and hence may contain not only tmc constructs, but also C code. The basic model is a state machine, although I have taken some liberties in defining what a state is for the convenience of the users.
Finally, appgen is a procedure for pulling everything together. It reads a simple specification file and produces a not-so-simple Makefile to control the compilation of all the pieces required to run the instrument.
The best source for documentation is the ARP Data Acquisition Web Page. From there, you can access these guides as well as manuals for many of the tools and release notes. As with the system as a whole, documentation is a work in progress. If you need to know about something and cannot locate documentation, feel free to ask me; you may prod me into writing the documentation or you may get a personal tutorial. If you don't ask, I won't know that you need help.
Copyright © 1998 by the President and Fellows of Harvard
College
Norton T. Allen