Saturday, August 20, 2022

"Move fast and break things" meets industrial facilities

This is my account of my epic quitting from CrossnoKaye. 

That is how it felt to me -- and how I imagine it will feel to people who are tired of bad bosses and of young tech elites connected to VC money who act like they are smarter than everyone else even when they can't afford it. My intent is for those fed up workers to know they too may have the power to show the bosses they need to treat people correctly, and if they don't, the information will come out.

* * *

"When you strike a king, you must kill him" -- Ralph Waldo Emerson

I worked at CrossnoKaye, a Santa Barbara, CA startup, as a Senior Software Engineer, from May 2020 to July 2022. The company describes itself as a "B2B SaaS company building a cloud-connected operating system that will revolutionize the heavy industrial sector". Software as a service in the public cloud is great for Netflix and is absolutely the wrong way to control an industrial facility where people can get hurt by unstable software; but it is also how they wooed the VCs, which sets the stage for the drama. 

(The "operating system" angle is just an oddity coming from the physics Ph.D. founders, who use iOS to illustrate how their software would run on industrial facilities that have little in common.)

The good: 

  • Opportunity to learn about heavy industries
  • Friendly people (except for the management when challenged)
  • Modern tools and technologies

The bad:

  • Harvard grads and brogrammers calling the shots where experience is critical
  • Avoidance of open discussion on serious issues
  • 40+ employees in a startup with no product-market fit

The ugly:

  • The founders serve their idea to revolutionize the industry -- on terms sweet to VCs' ears -- instead of the people who they want to use their product.

The company tried to terminate my employment and failed.

Had I simply resigned without them striking first, it might not have been fair to air their dirty laundry, and had their strike succeeded, I might be dismissed as disgruntled. Because the company botched the termination, something only people assured they are always in the right could have done, they opened themselves up for what follows.

* * *

On the morning of Friday July 15, I joined a meeting listed as a 1:1 with a manager, expecting to talk about a team issue I had been raising for weeks. Instead, the CTO and the COO were there along with the manager who said we are parting ways effective immediately, added it was nice knowing you and signed off. The COO said he was there only as an HR and that the company is offering 4 weeks severance. I confirmed to the CTO that my personal email they have on file is correct and said I am not interested in their offer. When I started telling him what had been going on, he said I am taking notes. After a couple of minutes of that, the COO said let's talk on Monday.

Our contract states it may be terminated by either side upon delivery of written notice. By Sunday evening I had not received one, so I emailed the CTO this resignation notice:

From: Davor Magdic <*******> 

Date: Sun, Jul 17, 2022, 11:51 PM

Subject: Davor Magdic's resignation from Crossno Kaye

To: Jesse Crossno <*******>

Dear Jesse,

I am hereby notifying you that I am resigning from Crossno Kaye, effective immediately.

Regardless of what Perry may have said on Friday, you and Erik asked me to meet to talk with you on Monday for unspecified reasons. Since I have not received any written notice from the company and I am still legally employed with the company, I am letting you know that I am choosing to end our business relationship unconditionally, right now.

I have many reasons for wanting to end my employment, the first and foremost being that I have seen enough evidence to believe the company is recklessly endangering the national food chain and I do not want to be a part of it, especially after two years of my repeated, ignored and actively suppressed attempts to bring the company’s attention to the fatal flaws in our ill conceived Atlas architecture.

This is particularly a concern as CK is now moving into B********* [facility] with its "kill plant" where Atlas software may not only cause food spoilage but may threaten life and limb. Two weeks ago, by merely turning on W********** [facility] agents for reading, all other facility alerts malfunctioned and we had to turn W********** off.

This completely unnecessary, architecturally wrong interdependence of facilities and overdependence on the cloud has caused many incidents in the past: when we turned on R******** [facilty] for read only, O***** [facility] control stopped working. When we fixed O******, R********* stopped reading. Users are all too frequently unable to log in to Atlas to perform essential duties only because of the misguided cloud-centric architecture. When N***** [facility] power outage happened, the agent wiped out all of its data, including OAS I/O licenses (for which the OAS software was entirely unfairly blamed when the fault was squarely on our end, using experimental and now unsupported k3OS software for critical production facilities), because the agent was looking to the cloud first. The next day, the exact same thing happened in R********.

When I brought up our architectural design issues on Slack back in March, Adam, Perry, and yourself actively suppressed further discussion at the company level, according to Perry, who first promised to D*** and me a discussion on the issue in DM, which I then announced in the channel, only to have it cancelled next week by Perry saying "I fear if not framed correctly it will seed the perception that we're going in the wrong direction as a company". Adam claimed it was him who told Perry to cancel it, Perry said it was first him and then you who decided to cancel it. This cancellation happened despite my repeated insistence that the worst thing we can do about a problem is not talk about it. All Ops people stood strongly in favor of decoupling the agent from the cloud as I suggested -- turning Atlas from cloud-controlled to cloud-assisted -- and C**** later reused my slides to rekindle the discussion, which to my knowledge still has not gone anywhere.

In addition to suppressing architectural discussions, I have seen what looked like an intentional effort to mislead investors. In the practice pitch for the second round of funding you and Bryan gave before the company, when you touched on user referencability -- a key metric for the Product-Market Fit -- you just said "our users love Atlas" and moved on the next slide despite the fact that our Net Promoter Score was only * [number]  -- our users were actually detractors. Whether that pitch was just for the employees or for the investors as well I do not know, I hope they were told the truth. When P*** [user] in O***** [facility] is recorded on the video not being able to log in to Atlas saying "Here I’ll switch to Atlas platform, or dashboard, or whatever the hell you want to call it", it is clear Atlas is hardly referencable from the user’s view.

The pattern of ignoring problems exists elsewhere in the company. When N****** gave a tech talk on B**** S********* [procedure], indicating that a PMF hypothesis is that customers will want to buy Atlas if it can save them energy and proposing to do this through ****** ******** ****** ***** **** ***** [procedure], I asked if we know if that can affect food quality and no one knew or has reported to have looked into since. When in a DM to N****** I pointed out research that shows high fluctuations in frozen food temperature degrade the quality of the food, with +-5C degrading dough and potatoes significantly (ice crystals form) and just +-2C degrading frozen beef with thaw loss and other undesirable changes, I asked him what temperature fluctuations we expect during B**** S********** and never received a response.

This is to say nothing of the team dynamic problems caused by the bullying behavior of team members and Perry’s inability or unwillingness to resolve it. You are aware of all the damage caused by Drew and how only after my repeated efforts to protect myself and others from his behavior Perry was forced to ask Drew to apologize which led to other people complaining and then Drew being fired -- only to result in Perry effectively reprimanding me for doing what he failed to do. Now a similar situation emerged with Adam, who engaged in disrespectful and manipulative behavior and even badmouthed D*** and C**** to me to try to diminish my pointing out that they too believe we are too focused on the cloud. The attempts at strongarming, disrespect and dishonesty became so unbearable with Perry allowing or condoning it that I felt I had no choice but to ask that he resolve this, which he never did. Of course I continued working with Ops and other people, even through a national holiday, to enable W********** and kept monitoring the facilities and providing support. I also kept meeting with M**** to help him with onboarding -- I was against him being hired for his own protection as I would not have recommended to anyone to work with Roman and Adam whose disrespect I documented in detail in DM to Perry.

The very fact that you didn’t know what was going on, on your own admission, shows that my concerns about Perry retaliating were justified.

I will only add this: four months into my employment, J**** C********* reached out to me to meet for coffee. He talked about enjoying not working at all -- there was never a project he said he was leaving CK for. He told me he left the company because his sense was that it was playing fast and loose with the safety and reliability of critical systems.

In retrospect, I wish I had listened to him and quit earlier myself, but it is never too late.




Their response Monday and in the days that followed seemed one of confusion and panic. First they insisted they terminated me first (and threw in a self-protecting "we take safety very seriously"), then wavered when I quoted the contract termination clause, then went back the next day to insisting it was them, sending me multiple notices on their own and asking me to disregard previous ones, I assume to escape filing my notice as a legal document that investors might see. I did not ask anything of the company, and by the terms of the contract their acceptance of my resignation is not required.

For the record, two months earlier, on my second annual review, I received great reviews from coworkers and 11% raise. (On my first annual review, where my manager gave me the negative feedback for defending myself, I received equally good reviews and 3% raise.)

If everyone is respectful and I disagree with the company direction, I wish them good luck and leave. But if someone in a position of power disrespects me repeatedly and the management allows it, I don’t leave; I have done nothing wrong. Since they decided to just get rid of me and couldn’t even do that -- who goes to terminate someone unprepared? -- I believe it fair to make my experience public.

A version of this letter has also been posted on the company's Glassdoor.

* * *

"Davor should have handled Drew through management."

That is how Perry, my boss, effectively reprimanded me in the form of "anonymous" negative feedback which he used to close my annual review held two weeks after Drew, the bully, was fired following my public confrontation with him after months of management inaction.

I was ready to forgive them for not protecting me and others from Drew -- everyone was anxious around the guy, even afraid to use words Drew didn't like such as "integration tests" and "drivers", as if Drew could wish them into a cornfield -- until Perry made this comment. To add insult to injury, Perry hid behind the anonymity (for the receiver) of the feedback process: a non-management person could not have known how much I engaged the management.

Drew had been praised by the management as brilliant, was an inside connection, and was immediately taken into leadership role, without even going through a coding test everyone else had to pass.

A month into the job he interrupts my report at a standup, unmentioned, unprovoked, with "why the fuck would you do that?!" I respond back with a small zing. ("Because in theory, theory and practice are the same, but in practice they are not", I said, but the only correct response would have been "Drew, don't talk to me like that". I was too shocked, as were others; I had to learn on the job.) When the team lead said let's all act respectfully, Drew said “You mean like not saying fuck?”, raised two thumbs up and grinned: “Fucking A!”

You get the idea what kind of person you're dealing with.

In the months that followed I and others experienced variations of that, but the management kept praising him, making him seem untouchable. After all my reports to the management, when upon another incident I confronted Drew in public, Perry tried to hush everything up. People DMed me with thumbs up and support and said in their previous jobs Drew would have been fired on the spot. 

At that point Perry had no choice but to ask Drew to apologize; I insisted the apology had to be made before the same people where the offense was made. This is how it went: at the standup Perry asks Drew "please, Drew, if you'd like, would you make an apology to J***". HR and CTO are present. Drew apologizes with "I didn't know I was doing anything wrong, next time, if I am, tell me." Perry asks Drew to apologize to me too, but Drew doesn't, which Perry does not address. Perry then praises Drew how great he is to work with and how much he has contributed to the company.

(What Drew actually did, in the words of one engineer, no one knew -- Drew actually blocked all progress on testing the critical piece of software for reasons only known to him.) 

When a company advertises a job, it is implied that they nurture a healthy, respectful work environment and deal with people like Drew swiftly. Had they said "must be able to work in a toxic environment and not push back when attacked by aggressive personalities in team lead positions", they would have had to offer a very different pay.

"Move fast and break things" meets industrial facilities

This is my account of my epic quitting from CrossnoKaye.  That is how it felt to me -- and how I imagine it will feel to people who are tire...