The Early Days: Part III

It was January of 2009, and I was looking for a job. My sister worked for this particular company and could help me get an interview for an entry-level position in the billing department at a Revenue Cycle Management company. Luckily, it paid off, and I interviewed with the EVP of Billing Operations. This person had been a massive fan of my sister and saw some of the same qualities in me. Suffice it to say, I got the job as an administrative assistant to the EVP of Billing Operations. I was attending school at the time for something completely unrelated and was only able to work part-time. I quickly became known as the go-to guy for computer issues in the billing department when the desktop support department refused to assist promptly.
As I mentioned earlier, I have some skills in troubleshooting computer problems. I was by no means an expert; I just came with this sort of tenacity to never quit until it was either working or needed additional resources to get working. For example, someone was having trouble adding a new printer and the appropriate print drivers. I had never done this previously, but decided, what the heck, I’ll give it a try. The first thing I did was open the Printers and Faxes menu from the Start menu, which is linked from the Control Panel. Mind you, this was Windows Server 2003 R2 because the company had not invested in upgrading the OS. I clicked Add a Printer and followed the wizard for a network printer. Since I knew nothing about print servers, I looked up a printer in the directory and clicked Find Now. I asked the person next to me what the printer’s name is. They replied that it’s the DM printer. Luckily, whoever set up the printer on the print server put the initials ‘DM’ next to the printer’s actual name. I clicked that printer, and since someone else on the terminal server already had the drivers installed, it was ready to go. I set this printer as the default for this person. That person was so grateful that I helped. This would be the beginning of my career in information systems.

I quickly built a reputation for fixing problems in the billing department. Word spread to the actual desktop support department, who then, instead of being furious that I was doing their job, gave me more tasks. This time, I needed to open XML files in a text editor like Notepad++ and identify issues with specific fields. When I found a problem, I fixed it and resubmitted the XMLs so that the system could load the values into the database. Well, I got really good at this task. Soon, I developed a relationship with the support department. The systems administrator used to say quite frequently that I could work in the IT department because of my obvious skill sets. This would come to fruition sooner than I ever anticipated. A member of the help desk team was out on medical leave and needed assistance in the department to close tickets, answer phone calls, and escalate trouble tickets, etc. The CEO of the company came to me with the help desk manager on that very proposition. This would be the description that shaped my career.

Soon after, I began working in the help desk department, and on my very first day, about 10 minutes into the shift, I was instructed to answer the next call. “No sweat, I got this,” I said to myself. The phone rang, and I answered it. The client on the phone began talking about something as if he knew exactly what was being said. I didn’t panic; I asked if I could put them on hold. I put the phone down, scrambled to the other help desk person next to me, and began repeating the exact phrases the person on hold had told me. The help desk person explained what they were talking about and how to fix it, but I would need to get on the phone to ask for more details about their issue. I repeated exactly what the help desk rep told me to say back to the person on the phone, and then I got a whole new slew of information I knew nothing about. Finally, the issue was resolved by my changing something in the application that the user had been experiencing problems with, and the phone call ended. After the phone call, I had like a hundred questions, and I asked the other rep, who was kind enough to explain it to me. From then on, every phone call was like this until I began to understand the systems I was trying to support well enough. After about 6 months of this, I was really good at troubleshooting almost any case that came my way. Then, I got my big break.

After about a year as a support representative, the former help desk rep who helped me transition into a system administrator. But after doing that job for 6 months, the systems administrator quit, and I was left to pick up the pieces. Fortunately, I knew the system and infrastructure pretty well by that point, so I wasn’t too worried until the CIO decided to hire a systems administrator with no real experience other than an MBA. Right? What does having an MBA do with systems administration? Anyways, I ended up spending about 2 months getting the new systems administrator up to speed, only for that person to leave for a new job. That’s when the system started crapping out, and I could no longer fix anything without adding additional hardware to the virtualization platform we were using, which was vSphere 4.0 at the time. I could only do so much, but we were over-provisioned on CPU and Memory, and our production servers crapped out multiple times throughout the day. Our clients were pissed off and threatened to leave us. I went around the CIO because he wouldn’t listen to anything I had to say, and I told the CEO what we needed.
More ESX hosts. We have four brand-new ESX hosts at our DR site, but the site wasn’t even active for failovers yet. I suggested that we bring back at least one ESX host to help with load balancing, since we were overprovisioned on both CPU and memory. The CEO listened to me, even against the CIO’s decision not to listen to me. We shipped back on the IBM x3690 X5 so that we now had two of these ESX hosts. The problem was that I didn’t know how to bring it online. We consulted with a company that assisted with our Dell Compellent SAN to help bring this ESX host online. It took the engineer, the CIO, both of our old systems administrators, and me 6 hours to bring this ESX host online. We worked until 2:00 am to get it done. The next day, another engineer from the same company came by around 9:00 to conduct a complete system analysis. The engineer was the Matt Damon Good Will Hunting of networking. He sat down at my desk and, in about 2 hours, created the entire topology of our system. In that time, he figured out exactly how our systems work. This took me several months to figure out, and I don’t think the previous systems administrator ever figured out how our systems work. This person changed my life.

Over the next several months, I worked with this engineer to resolve all of our networking, system, and virtualization problems. I learned more during those months than I ever thought possible. During this time, my boss, the CIO, finally appointed me as a systems administrator, but he still didn’t trust me. He even changed the title to infrastructure administrator because he felt like he knew more about infrastructure than systems and networking. Turns out, when we were trying to get this host online a few months back, the old system administrator convinced the CIO to allow me to be the “infrastructure administrator.” So, since 2012, that’s been my role. I eventually got my title changed to systems administrator, which took a lot of convincing. In 2013, the CIO was transferred out of the information technology department and placed into the department that was most fitting, sales. Turns out the former CIO had a background in selling computer memory, and he bullshitted his way into the VP of Information Technology by convincing another individual who knew even less about technology. From the time I first became the “infrastructure admin” until the day the CIO left IT, this individual tried to make my life a living hell. I guess I deserved a lot of it because that’s what happens when you know more about how the systems work than what the CIO claimed to know. I’ve wasted enough energy talking about this individual, so I’ll leave it at that. Now, I am running two data centers, and everything is working as it should. I’ve learned a great deal about systems, infrastructure, storage, and networking, which is why I’m going to be posting articles that have helped me configure VMware, Dell Compellent, IBM X systems, HP networking equipment, and Cisco hardware. Hopefully, my knowledge and experience can help others in situations like mine. Keep coming back for more content.

Leave a comment