Gerovitch: When do you first start working with onboard
After that, our firm turned to orbital stations. We placed the same computer on board the Salyut-1 station. The name of the station here coincided with the name of the machine. A major drawback was that the machine was not very powerful, and it required a lot of manual work. Cosmonauts had to take measurements and enter the data; even during computation they had to help the machine: the memory was so small that they had to enter intermediate results of calculations. On board the Salyut-2 and the Salyut-3 stations, the Salyut-2M computers were installed. Those were also made in Zelenograd. The Salyut-2M was a relatively advanced computing machine. It had a tape drive, made at a firm located at Novaia street (now it is called Kosmopribor). Magnetic tape for this drive was taken from American balloons. The Americans sent spy balloons over the Soviet territory; some of them fell down, and tapes were taken from those balloons.
We faced the following problems. First, other onboard equipment was poorly suited for working alongside a computer. The concept of electromagnetic compatibility had not been developed yet. Second, since everything was done very quickly, the software was not tested well. There were many malfunction reports which were difficult to sort out - whether this was a hardware or a software error. For the Salyut-4 station, we rewrote the Salyut-2M operating system completely. This station had an advanced navigational system, solar sensors (which timed the moments of entering and leaving a shadow), a Doppler velocity meter (on the Soviet territory there were transmitters, which generated a precise, stable base frequency, and there was a receiver on board, which measured the velocity based on the Doppler effect). On the Salyut-4 station, the trajectory was calculated automatically on board. As it turned out, however, there was no big need for that, since the ground control calculated everything precisely enough. The onboard system, in fact, served as a backup.
The main purpose for which this machine was actually used was displaying information for the crew: when will be the next communication session, how long will it be, when the spacecraft will enter a shadow, and so on. As for various measurement and calculations, those were not used but served as a backup, for information purposes only. This machine was not turned off for months in a row, and it operated normally.
On the Salyut-7, the last Salyut station, despite many breakdowns on board, the computer worked until the last minute, until power was shut off. There was also many computer malfunction reports - not clear whether software or hardware errors. Diagnostics at that time was not developed well enough. If there was a suspicion that it was a hardware error and pressure was applied on computer designers, they could not firmly reject this hypothesis. Especially since this machine had no redundancy. There was only a "cold" backup on board: normally the first machine operated, and only if it broke down, the second one would be turned on.
In the 1960s, in the Control System Division, headed by Boris Chertok, there were two departments, one led by Vladimir Branets, and the other by Ernst Gaushus. Initially it was planned that Branets would be working on cargo ships, and Gaushus on orbital stations. Branets' department preferred Argon onboard computers, and Gaushus' department Salyut computers. Based on the experience with the Salyut-2M, a blueprint for the Salyut-4 computer was created. In the early 1970s, in Chertok's division there were constant debates which machine to choose. We insisted that the Salyut-4 must be installed on all space vehicles, and Branets pushed the Argon. The Zelenograd firm built the Salyut-4, but then a "pause" occurred, and its production was delayed. More funds were needed. In the end, funds were not allocated, and the Zelenograd firm began the development of the Salyut-4 on its own.
In parallel with the Salyut stations, work on the Mir station began. Since Gaushus' department was overloaded, this project was assigned to Branets. Branets' department had more or less finished the cargo ship, and people in his department had time to do a new project. It was planned that the Mir would fly in orbital orientation, and in order to point equipment toward the Earth and the stars, a rotating platform was made in Czechoslovakia (this work was supervised by the Institute of Space Research). Wherever the station was, it could monitor a specific spot on Earth. Gaushus' department received an assignment to design a control system for this platform. This was in the mid-1980s. At the same time the Central Specialized Design Bureau in Kuibyshev, headed by Dmitrii Kozlov, was developing a new series of satellites, and together with the Zelenograd Science Center they decided to build a large computer complex: a high-speed computing machine with a single Manchester cable, which connected to individual pieces of equipment through peripheral exchange adapters (PEAs). The Zelenograd firm built this PEA. It was, in fact, a real computer weighing about 15-16 kg. It was planned that each system, for example, the engine control system, would have its own PEA, which would transfer data into the main computer. We decided to use this PEA as a control computer for the rotating platform on the Mir station, and this was done. As it turned out, I have never seen a better computer. It worked the whole time when the Mir was on the orbit. The Mir station itself was equipped with Argon-16 computers, but since those were not powerful enough, Branets designed the onboard network in such a way that the Argons could later be replaced with Salyut-4 computers, which were under development at that time. Branets was responsible for the Mir control system; they used both the Argon-16 and the Salyut-4 computers, and they had to swap the computers all the time. One day they shipped one computer to the station, another day another computer; there were constant malfunctions. And the first PEA that we installed worked on the Mir all the time without a single malfunction. Later other PEAs were added to it, as the functions expanded and other pieces of equipment were added, but this first PEA remained the central part of this system.
I am flabbergasted to see how ungrateful sometimes people are. Nobody thanked the chief designer of this PEA, Anatolii Shchagin. They could have written a newspaper article about him or given him a medal. In my opinion, there has never been such a computer. It had triple redundancy and was built with 16-layer ceramic boards. It did not have a single malfunction during the entire operation period.
Later on, in parallel with the Mir station, we started working on the Buran. There was a large onboard computer installed on the Buran. Whereas in all previous projects all software was written by a separate department, on the Buran everybody wrote their own piece and tested it as they could. The job of Gaushus' department was to put everything together and test it. In reality, the tests were somewhat superficial. It was a miracle that the Buran's flight was successful. They were right that they did not launch it the second time. It is well known that most problems occur at junction points: one did not say it right, another did not understand it right, and so forth. Even though there were special instructions given on how to avoid such situations, nobody could give assurances.
Gerovitch: After the Zelenograd firm handed the Salyut-1 over
to you, did they do any programming for this machine? Did they do any
modifications of hardware?
Gerovitch: Who was writing programs - engineers who understood
control problems or specialists in computing?
The situation with PEAs on the Mir station was different. PEAs already had a built-in operating system written by the Zelenograd firm. It had a debugging system, and programming was done in a machine-oriented language. One could use high-level languages, for example, Basic. In this case, the person who wrote the algorithm also wrote the program. The programmers also helped, but they largely worked on data exchange problems.
The Mir had a distributed complex. One PEA was installed on board, and there was a digital communication channel with the ground control. On the ground, there was a PEA and a personal computer next to it. When an operator on board saw something interesting on Earth, he could turn the rotating platform and point it in the desired direction, and the platform would then track this spot. The same could be done from the ground. Within the visibility zone, the ground and the onboard systems worked in parallel and in coordination with each other. Once the ground control was performed from Australia, later they took it to Europe, but this was at the time when I already left. This was a very good computer, but expensive: it cost about 100,000 rubles.
In Zelenograd, they had a large computer design and manufacturing complex. One organization was at the top, and it had many divisions. Different Salyut models was made by different people, but they all bear the same name.
Gerovitch: There were both Argon-16 and Salyut-4 computer on
board the Mir station. Which of them kept breaking down?
Gerovitch: You mentioned that the first Salyut computers
performed largely a backup function in case ground control fails.
Gerovitch: Were those computers actually used?
Gerovitch: Did ever a situation occur when control had to be
switched from the ground to the onboard computer?
Gerovitch: When did onboard computer begin performing
Gerovitch: On the Soyuz-TK, the onboard computer was an
essential part of the control system. What would happen if it broke
Gerovitch: In the early 1960s, was the decision to place a
computer on board an obvious solution or was it controversial?
Gerovitch: Were there any doubts whether this technology
proves reliable in flight?
Gerovitch: What problems arose in connection with software
Documentation for the Salyut-2M was improved, and the PEAs had complete documentation, which had only one error related to temporal functions. This error was noticed during ground tests. Since these computers worked in real time, several times per second they received exact time signals and also clock frequency (1,000 Hz) signals. The operating system had certain tasks to turn something on at specific moments. We noticed that some of these tasks were being simply skipped and were not carried out. The machine was malfunctioning! We called the chief designer in. It turned out, the operating system had a bug. The exact time signals and the clock frequency signals arrived in the system asynchronously from different generators. The machine, however, expected them to be synchronous, and for this reason it skipped. When they tested it on their simulator, both signals came from a single generator, and this glitch did not show. The documentation did not specify that the signals had to be synchronous, and we used different generators, which resulted in the malfunction. This was noticed during the ground tests, and we spent about a month on that. We had hard time figuring out why was it happening. Eventually they fixed it; they made a patch in the operating system. After that there was not a single problem.
Gerovitch: To what extent did the computer control cosmonauts'
actions? Did the machine set limits on what a cosmonaut could do?
Gerovitch: Could the cosmonaut ignore the limits set by the
machine? Did the machine only warn the cosmonaut of a potential mistake,
or did it firmly tell him that he must redo the task?
Gerovitch: That is, he could not do anything until the machine
accepts his data?
Gerovitch: And if, say, there was an error in the algorithm,
and correct data were rejected because of that, could he do anything
Gerovitch: Did you rely on machine more than on human being?
Gerovitch: Could the cosmonauts make any adjustments in the
machine so that this velocity range, for example, could be corrected?
Gerovitch: Have there been any cases when a cosmonaut on board
made program corrections based on instructions from the ground?
Gerovitch: That is, even though it was possible to correct
programs for the Salyut-2M on board, this possibility was never used,
Gerovitch: Were the cosmonauts taught the fundamentals of
computer programming, or just the operator functions?
Gerovitch: On the later models of space stations and
spacecraft, did the cosmonauts have functions that required a deeper
understanding of the functioning of computer?
Gerovitch: What were the arguments during the debate over which
machine to install - the Argon-16 or the Salyut-4? What were the advantages
and the disadvantages of one machine compared to the other?
Gerovitch: How did these machines compare in terms of weight
Gerovitch: In case of the Argon, was it "hot" redundancy? That is, did both computers work in parallel?
Zybin: To tell the truth, I do not know. I was immersed in algorithms and programs and did not care much about hardware. Gaushus and Sharov took active part in those debates.
Gerovitch: What methods were used to test and debug the
For the Buran, we also planned many tests, but we lacked time. The organization of work was complex: Chertok was responsible for the entire control system, while the core control system was developed by Pilyugin's firm. The latter designed a dispatcher machine. The integration of all software was assigned to us, that is, to Chertok. It was difficult to sort out who is responsible for what. On the one hand, Chertok was in charge; on the other, Pilyugin's firm was in charge. We developed a testing system, but there was not enough time, and we just muddled through. The deadline was approaching, and they pressured us. To test everything thoroughly, one needed probably two more years. It was impossible to tests everything [in the given time]. We managed to complete the tests of the regular sequence of operations, but nothing was tested for irregular sequences. We all had fears. Thankfully, everything ran on the regular sequence. There were no questions afterwards.
Gerovitch: Where there any attempts to write two parallel
programs for the same task to create software redundancy?
Gerovitch: Did you use American experience in any way?
For example, the Americans at that time widely used the Kalman method for measurement data processing. Both we and the Institute of Applied Mathematics thoroughly checked it, and it turned out that if measurement errors were large then it produced nonsense. We rejected it right away; we used the mean-square method and cut off everything extra. They did not do it; they acted directly. All the time we had the feeling that they were wrong or maybe they just published the wrong approach and we did not know what their real approach was.
Gerovitch: In software development it often happens that the
programmer keeps the details of the program in his head, and if he is
not available, then it is very difficult to understand the program.
Gerovitch: When was this programming discipline introduced?
Gerovitch: In his memoirs, Chertok describes cases when the
program author was absent and nobody could understand the program.
Gerovitch: Is Branets still a department head?
Gerovitch: Did the huge experience you
accumulated and the new methods you developed have any
circulation outside your department?
Gerovitch: Thank you very much for the interview.
site last updated 16 December 2002 by Slava Gerovitch