Plugging the holes in your wall

Building walls has been a human survival skill since our emergence as species. Establishing physical barriers to protect yourself from predators is very common in the animal kingdom, from the shells of the mollusks and turtles to the beaver lodges or the wasp nests, but no other species has ever shown the adaptability and versatility of human wall building. We build in all sizes and with all kinds of materials and it has been so important in history that it was probably a dispute over the placement of a partition wall what spurred the development of surveying in ancient Egypt, and the associated fields of geometry and mathematics.

In a time when life is becoming more and more digital by the day, the concept of wall building has also been translated to the electronic world. Famously the Great Firewall of China is intended almost as a one-to-one translation of its physical counterpart to protect Chinese citizens from "inadequate" content generated by foreign providers, but it is not the only one. Spam filters block the email we do not want to received, login pages prevent unauthorized access to digital assets, encryption helps keeping our communications private, the anti-viruses detect programs with strange behaviors, etc.

Photo: Bill Boaden

In the world of space instruments, we also need protective measures, because our instruments are delicate, unique and have to survive in a very harsh environment. Heat shields protect them from the Sun, thermal blankets from cold of space and other materials prevent the damage by high-energy radiation. The fact that space instruments are essentially impossible to repair (with Hubble Space Telescope being the one exception to this rule so far for very good reasons) also means that they have to be operationally robust: they shall not freeze and become unresponsive, and they shall not allow operations that can damage them, either directly or through wear-and-tear. One of my tasks has been to develop a tool that analyzes the operations sequences and identifies points where the plan violates the rules of operation that have been defined for the instrument. 

When I started in this job close to twenty years ago both the computational power and the number of parts that the instruments carried was significantly smaller, and that meant that the operational constraints where much simpler, mostly limited to switching on the instrument only if its temperature was in a certain acceptable range. But nowadays instrument have many configurable parts, some even with their own independent software, and keeping them properly configured and running in sync is far from trivial, so I pushed (and succeeded to convince) my boss to have a tool that would systematically check the operations plans.

Being as I am a firm believer in evolutionary prototyping as a development approach, the first incarnation of the tool was running within a couple of days. It was not able to check for many constraints, but it could already read the sequences and ensure that each operation was allowed sufficient time to complete before the next one was launched. Of course, those checks are only as good as the information that is provided to the algorithm. In engineering, this concept is called Garbage In, Garbage Out: if the information is wrong or incomplete, so will be the checks, but the duration of each operation was relatively easy to determine and the tool quickly demonstrated its usefulness.

The next steps were also relatively easy: you cannot command an element of the instrument if you have not switched it on or if you have already turned it off. This feature also detected a couple of situation where we had assumed that some parts of the instrument were going to be on already, but they were not.

The problem now is that, as we grow more and more used to the existence of the protective measures, we begin to forget that it is still in development, like a wall with some holes in it: it already provides a significant amount of protection, but there are still areas that are not fully covered. Ideally, Martin and I should sit together one morning and take the time to walk around the wall trying to find and plug as many holes as possible, but that is a lengthy and disagreeable job, particularly in the times of COVID-19 where this would have to be done via video conference.

My consolation is that all the critical cases, the ones that could potentially render the instrument unusable, have already been identified and the remaining ones would only lead to an occasional minor loss of scientific data, so hopefully we will manage to plug those wholes in the upcoming months and render the operations truly reliable. Have a nice evening.