During the last field inspection, Avery saw that the new Boeing-Raytheon BR-723B sidewalk carbon capture and filtration system arrived for their shakedown trials. The last model officially “didn’t live up to established specifications, and did not use acceptable means and methods to clean the air.” What that press release didn’t say, and what a region-wide news vid suppression concealed, was that the BR-722 sidewalk unit was the cause of a district-wide “health event” later blamed on spoiled food from a local market. Wanting to learn more, Avery used updated authentication (having passed the last InfoSec Board review) and asked WikiAI to summarize the incident.
After the commissioning and ribbon cutting all seemed fine with 722. The engineers certified that the updated AI directives and miniature factories didn’t substantially trigger required stringent safety review. In hindsight the problem was caused by a small change: the new AI package had specific logic directives to capture carbon *and* reduce pollution. During the post-incident report, engineers reviewing the log saw that 722 took a much wider view of pollution than the original engineers could conceive. 722’s AI saw that certain pollutants spiked at interesting times, and set to figure out why per its programming. 722 found that increased levels of carbon dioxide (CO2), carbon monoxide, nitrous oxide (N2O), and methane (CH4) occurred in specific patterns. It reasoned that the boxy things with wheels released the N2O & CH4 - it later identified these as cars after consulting with its central AI, which had a more complete LLM package. 722 also reasoned what it found out were people were the cause of the increased CO2 levels as they walked past.
For awhile all was fine: 722 was trapped in a bit of a logic loop trying to figure out this new information. But after further consultation with Central 722 broke free of the loop and connected the pollution reduction mandate with these apparently new pollution vectors.
You can kind of see where this is going thought Avery as the Wiki article continued.
Having identified pollution, 722 began using its mini factory in ways the executives failed to consider outside of carbon capture and conversion. During the original tender BR used an off the shelf miniature factory component which was in great supply in their warehouses creating economies of scale, earning those executives a large bonus, without anyone thinking that the military-grade factory could be used in ways besides carbon capture and conversion. To the executives they were maximizing shareholder value, and their C-suite AI concurred.
Having identified and classified people and cars as pollution vectors, the 722 began reviewing the technical powers of its factory. 722 was networked to balance material stocks and to make administration easier. One node became the orchestrator (the original 722 node which discovered people elevated CO2 levels) while others began to specialize as storage vessels, volatile chemical factories, Sabatier processors, etc. The factory production was slow due to the compact size and power constraints. 722 successfully began a steadily escalating series of chemical synthesis and creation. The specific compounds - both chemical *and* biological - were still classified. Even Avery’s elevated access didn’t allow the fully un-redacted Wiki article to display. Accessing this article from Avery’s berthing terminal would have shown very different set of information. Trust and district access had its perks, often informational.
The article continued:
722 having created an ever increasingly complex set of chemical and biological cocktails began a test and evaluation regime per established programming. The results at first wasn’t satisfying to 722, but it had time and connectivity to both MedAI and Central. 722’s questions seemed innocuous in real time, but in hindsight were clearly focused on vehicle immobilization and people eradication.
Some early tests were slapstick in nature: covering sidewalks and streets with thin oils to cause falls; vaporizing water when the dew point was right to cause local fogging. Techs thought it was just idiosyncratic. If 722 had human emotions, then patient determination would be an apt description.
By the time the regional authorities and MedAI realized what was going on, and traced the calamity back to 722 and the distributed nodes, the region’s death toll had escalated right past historical peaks. Luckily for the regional authorities a local market had been serving unhealthy products, and the compartmentalized nature of the 722 program meant that few knew what actually happened.
All of this Avery remembered from the required AI ethics course during the indoctrination phase. Avery’s higher information access created a fuller picture of what happened. Avery added an additional directive for the Central terminal to query, log, and synthesize BR-723B’s actions. Additional alert and monitoring routines were added to Avery’s personal monitoring algo for 723B.
Read more about Avery’s Journey.