Blog Posts and News from the Mansbach Lab

Like many PIs, I am perpetually in search of sources of funding, especially right now, given everything that’s happening in the U.S. (I am fortunate to be in Canada, but I have collaborators in the States, so while I am certainly less impacted by the goings-on, I am still impacted.) I happened to come across a large company whose program offered a fairly substantial sum of research funding for about a year. It did seem to align with my research interests, so I took a slightly deeper look, including at the agreement for if nominated to receive this funding.  Some of this was pretty standard stuff—coming to an intellectual property agreement, providing quarterly reports, etc.  And then we came to the section on Open Source Software.  Now, I do need to disclaim that I don’t know if this is standard or not (I wouldn’t really be surprised if it is).  

The section on Open Source Software was pretty brief, noting only that one should only use or create software with a license listed by the Open Source Initiative and should not use or create software subject to a list of specific licenses, including… “Any of the JSON ‘do no evil’ licenses.” 

What. 

I had to read that again.  I’m usually a pretty straightforward open source person, so I primarily stick to the MIT license, just because that’s what I know. A bit of googling around informed me that the JSON ‘do no evil’ license is, let’s say…contentious in the field of Open Source Software, as explained, for example, in this article, because it includes the phrase, “This software shall be used for Good, not Evil.” 

Now listen, I get that “Good” and “Evil” aren’t defined in the license, and I even understand why that might give a legal team a headache. But on the other hand—really? It’s being referred to as “troublesome,” a “fly in the ointment,” and the simple explanation is, “Actually, defined or not, this clause is enough to disqualify the JSON License as an open source license per se. Point 6 of the Open Source Initiative (OSI) definition of an open source license is “No Discrimination Against Fields of Endeavour,” which would include evil ones.” 

But…typically we do try to discriminate against evil fields of endeavor, and for good reason. (The old “paradox of tolerance” in which you can tolerate anything but intolerance.) The way this is phrased is tongue-in-cheek, of course, because the whole thing is tongue-in-cheek (notably, IBM has an exemption from the Do-No-Evil Clause). 

Should it be, though? 

Yes, “good” and “evil” are ambiguous concepts at best, and, yes, a focus on a binaristic good/evil divide is rather Western-centric. But at the same time, the STEM fields in general, and software engineering in specific, are absolutely not exempt from moral implications. What’s worse, they have a long and storied history of trying to pretend that they are. Computers are “unbiased” (no, they’re not [1]), algorithms never make mistakes (yes they do), algorithms certainly don’t cheat (yes they do), large language models aren’t bullshit artists (yes, they are [2]). 

There is a tendency to believe that if something isn’t easily and clearly measurable, it isn’t the purview of software development. This leads to an old boys’ club mentality, to a desire to go-fast-break-shit, and to laugh and joke about pesky useless indefinable notions of moral integrity and good and evil. 

But we, as a civilization, do define attempt to define moral integrity and we do attempt to define good and evil—at least to the extent of having laws that generally attempt to enforce some level of good. Those laws aren’t always beneficial; sometimes they’re pretty harmful. It’s still better than not trying at all. 

But how can we possibly correctly assess the moral value of our work? Surely it isn’t possible to do, since, after all, “good” and “evil” are such grey, loaded, complex concepts.  Wouldn’t it be better just to stick to things we can measure? Why not just build a language model that’s only trained on slurs? We can’t measure emotional pain. 

I think we all know by now that my answer is going to be a resounding NO. Not all questions have a single, correct answer; some questions don’t have answers at all.  That doesn’t mean there isn’t value in the act of asking them. Of course you should consider the ramifications of a language model that spews hate speech—and I don’t just mean the optics of it.¹ 

So maybe: don’t laugh at the question. Don’t act like you’re above asking what the impact of your work will be, and whether it will cause harm.  There is no shortcut for this. Morality, integrity, and harm are all questions of context. Questions of history, philosophy, and ethics, and similar apply differently under different circumstances. 

There is no one right answer to every question.  The question is still important. I’m not expecting software engineers to personally solve the ills of the world. Hell, I even agree that sometimes you do need to use that software for evil—there is such a thing as a ‘necessary evil,’ after all. I just think that maybe it’s worth taking an hour or two to decide whether your software usage is good, evil, or morally neutral, and writing up a paragraph justification. 

And then possibly I won’t click in to see that a big corporation is explicitly telling me that to work with them I have to be all right with not using a ‘do no evil’ software license without a trace of self-awareness or irony. 

(Anyone remember when Google’s motto was “don’t be evil”? Yeah, me neither…) 

References and Acknowledgements

[1] O’Neil, Cathy. Weapons of math destruction: How big data increases inequality and threatens democracy. Crown, 2017 

[2] Hicks, Michael Townsen, James Humphries, and Joe Slater. “ChatGPT is bullshit.” Ethics and Information Technology 26.2 (2024): 1-10. 

With thanks to Mari Magen for invaluable feedback 

Large Language Models (LLMs) have been thrust aggressively into the public sphere lately, along with other forms of “generative AI¹,” and have generated quite a bit of conflict. Many creators have loudly decried the way that most of these LLMs/genAI models are trained by stealing (scraping) their work, while others have adopted them as tools that allow them to better express themselves.

I was talking to a good acquaintance the other day, when he waxed eloquent about how useful these models were for him or for others who had disabilities that made it particularly difficult for them to create writing or visual art. The conversation, like many conversations surrounding the topic, was quite fraught, and at one point he expostulated (paraphrased), that he found it offensive that authors would try to stop other people from continuously thinking about and being inspired by their words. “That just makes you Anne Rice,” he said.

Some important context and background: my job is in research closely aligned to generative AI (although we primarily use the tools to try to discover new antibiotics in conjunction with other biophysical techniques), while one of my major hobbies is writing fanfiction. Fanfiction refers to writing stories about other authors’ characters or stories. Often these stories are posted on the Internet, on a site like Archive Of Our Own, where other people can read them for free. Anne Rice was the author of Interview with a Vampire (and many other novels), who is somewhat infamous in fanfiction circles for her aggressive legal pursuit of fanfiction writers, because she believed that it was inappropriate for others to write her characters into scenarios she didn’t agree with. So what my acquaintance was claiming was essentially that people publishing fanfiction who didn’t want their stories used to train LLM models was that they were unreasonably attempting to control how other people were inspired by their work, after making that work public for anyone to read. As a fanfiction writer myself, this would obviously make me a hypocrite.

My acquaintance was wrong: I’m not Anne Rice. I do understand how he got to this point, but it’s very important to me to clarify the distinction. In fact, one of the reasons it’s so important to me is that it’s taken me several years to put my own finger on the difference! When I was first introduced to the idea of training LLMs on text scraped from the internet or visual models on downloaded art, I didn’t understand why other creators were calling this “theft.” After all, we had posted these works publicly, and as an “AI”-aligned researcher myself, I knew that the model was in some sense just doing what a reader would do, but much faster: scanning through the work for patterns, which it could then use to produce text or images that were similarly-shaped to its inputs. The model doesn’t directly reproduce the work, nor does it store it², it uses it to update the internal function to better create things that seem similar.

So…what’s the difference?

The difference is in interpretation. When a person reads my work and is inspired by it—which I want them to be!—they may not interpret it in the way I intended them to. In fact, they may pick up a very different message from the one I put down. There’s a reason that some fans of Tolkien are feminists like me and others are white supremacists, a philosophy I abhor. We all come in to a work with our own contexts, experiences, and understandings, which shape how we interpret it. But that person still needs to read exactly what I wrote. When an LLM model is trained on my words, it does not come in with context, understanding, or interpretation. Even if the subtask it performs of trying to make human-like text is similar to a human author, it does not exist in the world outside of the internet. It does not have the context to understand what it’s reading. It only performs the subtask of making text like a human. It does not interpret³. So, when some other person uses the model to aid in expressing their own ideas, they don’t have to be influenced by mine at all. They can use my words to support any point, without having to actually engage with the ways I’ve used those words, and the stories I’ve tried to tell.

If someone with whom I have deep philosophical disagreements reads my fic and is inspired by it and writes a new story that comes from a place I view as reprehensible, I’m not going to be exactly happy about that. But they have at least listened to me before doing it. They basically have to. But the LLM breaks this contract. Now there’s no “listening” stage. A person can directly take my craft (admittedly it likely requires mine and many others) and stitch it together to support any viewpoint.

My words aren’t just there to sound like a human. They aren’t there to be chaff that supports any point. They are my voice. If you aren’t willing to listen to me in the first place, no, I don’t want you using my skill as part of your tool.

To my acquaintance: this isn’t listening and being inspired by me at all! This isn’t continuing to think about what I had to say! You never listened to me to begin with. I don’t need you to agree with me on fair and reasonable ways to use LLMs. I do need you to understand why I’m not Anne Rice.

Lab alumnus Sam Renaud supervised and co-authored this Space Concordia paper. The project investigated “the safety and feasibility of a launch of the StarSailor rocket from a prospective launch site in Churchill Manitoba, Canada. …Space Concordia aims to use the methods established in this manuscript as a general handbook and reference for future sounding rocket launches out of Canada.” Congratulations, Sam!

Full article : “Establishing the requirements for safe rocket launches with respect to weather”

The research project led by Ré Mansbach and Claudine Gauthier from the Department of Physics of Concordia University, has been granted $250,000 to explore the neurological alterations linked to long COVID.

Full article : “Concordia researchers receive $500K from Government of Canada to boost ‘high-risk, high-reward’ studies”