1.5 Case Studies: The Cost of Poor Communication

As an intern working for an established local business, Blake was fully expecting to get a permanent well-paying job within a few months. His supervisor asked him to do some background research on a key client project, and find some sources to support the proposal they would be presenting to the client next week. Although the company policy prohibited using commercial AI tools when dealing with client information, Blake thought he could save some time by using ChatGPT to do the research. He fed the information into ChatGPT as a prompt. ChatGPT generated a dozen sources that looked like exactly what he needed for the proposal, and even provided brief summaries and fully formatted bibliography. Blake was thrilled that what might have taken him many hours to complete was done so quickly! He sent the report to his supervisor, and called it day.

When he came in to work the next day, he was shocked to learn that he was fired! His supervisor had reviewed the research, and found some information that simply did not make sense to her as an expert in this field. In trying to follow up on the sources, she found that some of them did not exist! Others existed, but were not accurately summarized. A prominent local business owner had been quoted as saying something he had never said. Essentially, the research was useable, and if his supervisor had included it in her presentation to a client, she would have not only lost the account, but likely her job as well. Indeed, the company might have been sued, because Blake had violated the company privacy policy by sharing sensitive client information with a non-secure commercial AI, which then used the information as part of its training data.

The Ace Electric Company worked day and night to develop a new system for storing electricity generated by solar panels. They knew that, although the competition was fierce, their system could be store more power and do it more affordably than the competition, and could be easily adapted for a variety of business uses.

The owner, eager to capture the market, personally but somewhat hastily put together a 100-page proposal and sent it to the several major businesses  in the area, recommending that the new Ace solar collection and battery storage system be installed.

She devoted the first 75 pages of the proposal to the mathematical theory and engineering design behind this new system, and the next 25 to descriptions of the new assembly line she planned to set up to produce the necessary equipment quickly. Buried in an appendix were the test results that compared her system’s performance with competitor models, and a poorly drawn graph showed the potential cost savings over 3 years.

The proposals did not receive any response. Ace Electric didn’t get the contracts, despite having a superior product. Six months later, the company filed for bankruptcy.

Superb Software, Inc. had built a reputation for designing high-quality and user-friendly database and accounting programs for business and industry. When they decided to enter the word-processing market, their engineers designed an effective, versatile, and powerful program that the company felt sure would outperform any competitor.

To be sure that their new word-processing program was accurately documented, Superb tasked the senior program designer with writing the instruction manual. The result was a thorough, accurate and precise description of every detail of the program’s operation.

When Superb began marketing its new word processing software to businesses, cries for help flooded in from office workers who were so confused by the massive instruction manual that they couldn’t even find out how to turn the program on! Then several business journals reviewed the program and judged it “too complicated” and “difficult to learn.” After an impressive start, sales of the new word processing program plummeted.

Superb eventually put out a new, clearly written training guide that led new users step by step through introductory exercises and told them how to find commands quickly. User surveys reported very high satisfaction rates. But the rewrite cost Superb $350,000, a year’s lead in the market, and its reputation for producing easy-to-use business software.

Diane supervised 35 professionals in 5 city libraries. To cut the costs of unnecessary overtime, she issued this one-sentence memo to her staff:

After the 35 copies were sent out, Diane’s office received 25 phone calls asking what the memo meant. What the 10 people who didn’t call about the memo thought is uncertain. It took a week and several follow-up emails to clarify the new policy.

Kris was simultaneously enrolled in a university writing course and working as a co-op student at the Widget Manufacturing plant. As part of her co-op work experience, Kris shadowed her supervisor on a safety inspection of the plant, and was asked to write up the results of the inspection in a compliance memo. In the same week, Kris’s writing instructor assigned the class to write a narrative essay based on some personal experience. Kris, trying to be efficient, thought that the plant visit experience could provide the basis for her essay assignment as well.

She wrote the essay first, because she was used to writing essays and felt confident she could do a good job. She had never even seen a compliance memo, much less written one, so was not as confident about that task. She began the essay like this:

On June 1, 2018, I conducted a safety audit of the Widget Manufacturing plant in New City. The purpose of the audit was to ensure that all processes and activities in the plant adhere to safety and handling rules and policies outlined in the Workplace Safety Handbook and relevant government regulations. I was escorted on a 3-hour tour of the facility by supervisor and learned a lot of fascinating things…

Kris finished the essay and submitted it to her writing instructor. She then revised the essay slightly, keeping the introduction the same, and submitted it to her co-op supervisor. She did well on the essay, getting an B grade, but her supervisor told her that the report was unacceptable and would have to be rewritten – especially the beginning, which should have clearly indicated whether or not the plant was in compliance with safety regulations. Kris was aghast! She had never heard of putting the “conclusion” at the beginning. She had to miss the company softball game that Saturday so she could rewrite the report to the satisfaction of her supervisor.

Sam and Dav worked for weeks to perfect their pitch to their local community version of Dragon’s Den, hoping to get one of the judges interested in investing in their start up. They made sure that their presentation projected confidence and enthusiasm in their product, and designed flashy graphics to supplement their pitch. They promised their product would generate tremendous profits with astoundingly low overhead costs, provide amazing benefits for users, and take over a phenomenal percent of the market share at an unbeatable price point. When the first judge asked about current materials and promotional costs, Sam said they were “super low.” When the second judge asked about the reliability of supply chains and labour, Dav said they had “no worries.” The third judge starting asking about “return on investment,” but stopped part way through and said “never mind.” The remaining two judges did not bother to ask any further questions and did not make any offers to invest in their idea. Sam and Dav were shocked and disappointed that the pitch they worked so hard on garnered no interest whatsoever from the judges.

The following excerpt is from Carl Sagan’s book, The Demon-Haunted World: Science as a Candle in the Dark,^[7] itself both a plea for and an excellent example of clear scientific communication:

The Superconducting Supercollider (SSC) would have been the preeminent instrument on the planet for probing the fine structure of matter and the nature of the early Universe. Its price tag was $10 to $15 billion. It was cancelled by Congress in 1993 after about $2 billion had been spent — a worst of both worlds outcome. But this debate was not, I think, mainly about declining interest in the support of science. Few in Congress understood what modern high-energy accelerators are for. They are not for weapons. They have no practical applications. They are for something that is, worrisomely from the point of view of many, called “the theory of everything.” Explanations that involve entities called quarks, charm, flavor, color, etc., sound as if physicists are being cute. The whole thing has an aura, in the view of at least some Congresspeople I’ve talked to, of “nerds gone wild” — which I suppose is an uncharitable way of describing curiosity-based science. No one asked to pay for this had the foggiest idea of what a Higgs boson is. I’ve read some of the material intended to justify the SSC. At the very end, some of it wasn’t too bad, but there was nothing that really addressed what the project was about on a level accessible to bright but skeptical non-physicists. If physicists are asking for 10 or 15 billion dollars to build a machine that has no practical value, at the very least they should make an extremely serious effort, with dazzling graphics, metaphors, and capable use of the English language, to justify their proposal. More than financial mismanagement, budgetary constraints, and political incompetence, I think this is the key to the failure of the SSC.

A civil engineering firm was preparing a structural assessment for a small municipal infrastructure project: the retrofitting of a pedestrian bridge. The project had a tight deadline. The bridge was expected to open at the beginning of the tourist season. Budgets were constrained, and senior engineers were stretched across multiple projects.

To save time, the management encouraged junior staff to use generative AI tools to help draft technical reports, summarize standards, and explain calculations in clear language for non-technical users.

Jasmine, a newly hired junior civil engineer was responsible for drafting the structural assessment report which would be reviewed by city officials and used to decide whether the bridge can safely reopen. She used an AI tool to summarize load requirements, analyze fatigue and corrosion risks, and reference relevant engineering standards and prior use cases.

The AI produced confident, professional-sounding explanations and cited several standards and prior studies. Pressed for time, Jasmine integrated this material into the report without fully cross-checking the referenced standards or validating calculations.

During a later peer review, a senior engineer discovered that the report cited outdated standards,  referred to a case study that did not exist, and the descriptions were oversimplified and misrepresented load combinations and peak stresses.

The report was withdrawn before submission, delaying the project and triggering an internal review of AI usage and governance for the firm.

Mei is a mechanical engineer working for a mid-sized manufacturing firm that designs custom enclosures for industrial automation equipment. Her team is responsible for monitoring a newly designed electrical enclosure to ensure it remains within safe operating temperatures after its installation in a food processing facility. The project timeline is tight. Management expects a preliminary thermal assessment by the end of the week so procurement decisions can move forward.

Mei knows that running a full computational fluid dynamics (CFD) simulation would provide the most reliable results, but setting up the model and running multiple scenarios would take several days. Instead, she uses an AI tool to generate a written explanation of heat transfer and airflow patterns based on similar enclosure designs.

The AI output clearly explained how natural convection and ventilation openings should dissipate heat effectively under expected operating conditions. Mei incorporates this explanation into the report, along with basic temperature estimates derived from traditional calculations. It looked professional and technically sound. She did not include graphs or simulation outputs, but the language and tone closely resembled reports that normally accompany valid models.

When the project manager asked Mei why the overheating was not anticipated, she realized that these decision-makers assumed she had conducted a full CFD; the AI generated explanation did not provide sufficient data to allow them to make a decision and move the project forward.

David, a junior structural engineer, was preparing a design memo for a retaining wall in a residential development project. The project site is located in an area known for variable soil conditions, and geotechnical data had not yet been finalized. David composed a cautious memo emphasizing the assumptions and uncertainties around the soil data.

Before submitting the memo, he used an AI tool to improve the clarity and professionalism of the content. The tool rewrote several sections, refined his sentences and removed uncertain language. Pressed for time, David determined the revisions sounded professional and submitted the memo to the project manager.

The following day, the project manager tells David that the project is ready to proceed. David was surprised at how quickly the decision was made without addressing the uncertainties in the soil data he discussed but assumed that the experienced engineers shared his concerns and would correct the issues.

David intended the memo message to be provisional, but the AI tool edited the content to communicate certainty and confidence. Construction began early and when the soil tests later revealed weaker conditions, part of the work had to be removed and redesigned.