There and Back Again: An AI Career Journey

I had wide-eyed ambitions to be an Artificial Intelligence developer.

I studied all the latest material. By that I mean sci-fi movies — Terminator, War Games, RoboCop, and Aliens. Okay, yeah, those are a bit apocalyptic. But, TV shows like Knight Rider and Star Trek: The Next Generation made me believe anything was possible.

I had cut my teeth on the bleeding edge of technology known as the Atari 1200XL and was ready for AI.

I also enjoyed programming — I guess BASIC and Pascal counted back then. Despite this, I created my own games and even a database engine for a baseball card collection. My dad didn’t want to spend the money on software and put us kids through school, so he challenged me to write them.

In my senior year of high school, I wrote a really fun and interesting program that would generate images of the electron orbitals for the hydrogen atom using a complicated wave equation. One dot at a time, it would produce images. Not quite as nice as these here — it was still the 80’s, but you get the picture.

My physics teacher was impressed enough that he nominated me for a scholarship at Rensselaer Polytechnic Institute. This opened the door for me to get accepted into the school as a computer science major. I was well on my journey into AI.

AI Winter

At Rensselaer, our AI professor asked us to write a program to play solitaire, the card game. I went to work writing in LISP, only to find that the computer wasn’t really learning anything.

I was picturing Commander Data figuring out how to play the game, faster and faster with every shuffle. It shouldn’t be me programming it, telling it what to do every step of the way.

I asked the professor how this is considered “artificial intelligence”. She said our job as a developer is to tell it the best move to make at any time and be sure to cover all scenarios. Basically, transfer my so-called intelligence into the computer.

That was artificial intelligence in the early 90’s — heavy emphasis on artificial.

I could feel the bubble burst inside me, and I wasn’t alone. The technology simply wasn’t there yet. This turned out to be the Second AI Winter.

Aha Moment

I finished my studies and put my ambitions as an AI developer on the shelf. I started my career as a systems and network infrastructure administrator and I truly enjoyed the challenges that came with it.

Being a software developer at heart, I found ways to automate many of the repetitive administrative tasks to monitor system health. This came in handy since my users would frequently ask me why the network is so slow. Or the classic question, “Is the network down?”

Since my employer was loathe to spend any money, I went in search of open source software. But, you can only go so far without opening your wallet. Commercial solutions costing six-figures would get you there, but we weren’t willing to spend that. So, I sat down to write my own tools from scratch, emulating the expensive software.

Now, when someone asked if the network was down (spoiler alert: it was not), I could look in 5–6 different places and in about 10 minutes figure out exactly what (or who) was slowing things down.

I was excited that I now had the data to work with, but it became exhausting digging through it all.

Then came my “aha moment”.

The concept of Big Data was all the rage mid-2010s. I was attending a conference that gave a hypothetical example of someone finding the best way to get from point A to point B by train. In theory, the Big Data system could collect the train schedules, consider wait times, available seats, detours, ticket prices, etc., to suggest the best option.

Aha! This is just like my challenge of pulling bits and pieces of system information from different sources, then connecting the dots to create a clear picture.

All I needed to do was write another tool with the logic to piece it all together. Now when my users would ask their classic question, I could answer them with a few mouse clicks on a web server I built.

It felt like I was right back in college.

Learning

This sparked my latent computer science interest that was sitting on the shelf for a couple of decades.

I decided to pursue this new passion and signed up for data science and machine learning classes. My evenings and weekends were filled with as many online classes as I could take.

Looking back, I realize that the “aha moment” and revived passion were the real reason I grew tired of the infrastructure administration role. Don’t get me wrong — it is good work. But I found a better outlet for my skills, so every support call became unbearable.

It was time for a change.

Fortunately, much like my high school teacher that recognized my potential, my manager recommended me for another role in the company, appropriately titled Big Data Analyst.

I jumped into creating sales forecast modeling using an elegant programming language called R. Then I implemented neural networks on this model which forced me to rewrite it into Python. Ultimately, I used AutoML to find the best model, SHAP for explainability, and packaged this into Docker, plus integrated with Qlik Sense for visualization.

I don’t like doing something half-way.

Then the day came when my boss asked me to help our consumer service department implement an AI image classifier. Consumers would call for support on our products and we would spend a lot of time figuring out which product they have. A picture makes it easier, and the AI makes it faster.

The original intent was to find a third party company to create the AI for us, which would be six-figures expensive. I know how much our company is allergic to expenditures, so I offered to create the AI model myself. I set off collecting the image data, training the model, and creating a simple web interface to make it usable.

AI Spring

This is where my years of infrastructure experience came to the rescue. I had access to the internal resources, knew how to build a Linux server on Azure cloud to get the GPU accelerator, and could stand up a web server to make it accessible. This was 2019, and advances in hardware and software arrived at just the right time. At last, AI Spring was upon us.

A quick 30-minute demo to the head of the consumer service department persuaded him to dump all negotiations with the third party company. I was given the green light to take this on full time!

After 30 years, I reached a full circle. My AI developer dream became reality — without the killer robots, thankfully.

I spent the next 6 years expanding this application and is the highlight of my career. I even received our company’s Individual President’s Achievement Award. The project became very successful and I’m happy to share what I learned in a series of writings: