Contributing to a Political Campaign as a Nerd

As I promised in my previous politics article, I will continue not to advocate for specific politics and remain non-partisan on my blog. I do encourage everyone, regardless of beliefs or party, to participate in politics, because it affects you whether you participate or not. Participation in our democracy can only improve it.

Over the past year or two, I have come to feel far more strongly about my politics. I live in the United States, and it’s impossible to pay attention to the news and not feel some kind of way about our Republic. This year, I decided that I needed to contribute more directly, not just passively partake in politics. If you already follow me on twitter, you know that I wear my politics on my sleeve. Like many readers, I regularly vote. Like many readers, I donate to campaigns. That’s not enough for me anymore. So I decided to do more, and reached out to some local campaigns to find out what I could offer.

I will admit that this is scary. I have spent almost 20 years working on a relatively narrow field of expertise within IT. I had no experience with politics. Going from 20 years of experience to 0 is intimidating – but if I did it, so can you. If you read this and want to contribute, I want you to know that you will do just fine. As divisive and loud and argumentative and nasty as politics seems on the evening news, every group I have worked with over the past year has been welcoming, very graceful about my lack of knowledge and mistakes, and very accommodating to how much time I have available. Please, don’t let your fear keep you out. Reach out if you have any questions!

The expertise we have is sorely needed, though, especially in political campaigns. You will quickly find out that most people involved in political campaigns are not computer experts in any way. Sure, they’re computer savvy as most people are nowadays, but there are significant gaps in that knowledge that needs filled. All you need to do is read the news and you will quickly hear about campaigns that are hacked and crowdsourced analysis of what’s on a politicians phone and overwhelming numbers of twitter and facebook shenanigans. Your help is needed and will be welcomed.

I joined a campaign and I had no idea what I was getting into. I did not find many others in tech who have shared their experiences joining their first campaign. I hope this article helps fill this gap a little bit – and if any readers are in the same situation, I would encourage you to blog about it as well! Alright, let’s get volunteering!

As I hit publish on this, there are 85 days until Election Day in the US. It is NOT too late to volunteer! Your assistance will be welcomed up until the final moments on Election Day, and there are always future elections to prepare for.

What to expect

When you click Sign Up on a campaign web site, you’ll be offered some “normal” work – canvassing, phone banking, putting a sign in your yard, etc. All things technical are notably missing from the list. To offer your technical expertise, you will have to reach out to the campaign directly. Many campaigns or candidates have a listed phone number on their website. If not, try looking at the county or state party’s website for a phone number, and inform them that you would like to get in contact with the campaign.

You will have a chance at some point to talk to the candidate or a campaign manager and make sure that’s who you want to work for. Treat it like a job interview! The campaign will ask what you can do, and you get to ask the campaign about what they will do if elected. Be honest! When I first talked to my campaign, I explained that I had 20 years of IT experience but no campaign experience. I was willing to take on things unknown, but I wanted to make sure they knew it would be new to me. I found out they had an experienced webmaster who would be providing me assistance if I joined. I also asked a lot of pointed policy questions to ensure that I would be happy if this candidate was elected. Get your questions answered and let the campaign know whether you want to join and what you can contribute.

General tech tasks

There are so many areas you can contribute to the technology side of a campaign, regardless of where in technology you work. Here’s a very short, very incomplete list of items you can help with:

  • Setting up a free Slack and teaching people how to use it (EVERYONE uses slack nowadays!)
  • Setting up a website and analytics
  • Configuring multi-factor authentication on all services
  • Setting up apps on phones and tablets
  • Answering questions about how to use a computer, application, or service, even if you’re just functioning as Google as a Service for really busy people
  • Providing a sounding board for anything technical, including how technical people and companies may respond to something

Depending on what your expertise is, you may be able to offer some very specific needs. Surely, what you know can be applied to a campaign, though I may not be able to tell you how. A lot of my expertise is in information security. Here are some examples of InfoSec advice you can provide:

  • Explain threat models. Make sure you know how they apply, too; the threat model of running for US Senator is much different than that of running for a local council seat. Everyone can benefit from making sure they don’t expose their financial details to the world, but fewer are worried about specific attacks by enemy nation-states.
  • Ensure services are registered to a well protected account that belongs to the campaign instead of a random gmail account that belongs to someone who may leave the campaign.
  • Make sure MFA is enabled everywhere possible.
  • Restrict access to services to who needs it, and at the least permission level

Remember that you are advising a campaign, not running your own business, so you will probably “lose” some arguments in areas where you are objectively the expert, and that’s okay. Make sure everyone acknowledges the trade-offs being made, and do your best to minimize the potential fallout. If there’s a realistic chance of failure, prepare remediation plans so that you are ready if they are needed – the same kinds of thing you do at work to cover your company’s butt.

Be aware of how much you can contribute. If you can only spend 2 hours a week with the campaign at odd times of the day, maybe you are not the best person to run their web site. That’s okay, just make the campaign aware that you want to help as well as your limits and surely they can find a way to make it fit. If you can spend more hours, then I encourage you to take on more significant tasks. If your circumstances change, just let the campaign know!

There are tons of small things like this that you can contribute. Don’t worry if you can’t think of something now, if you reach out to the campaign I’m sure you can come up with something together!

Larger projects

In addition to these general tasks, you may be able to contribute to higher level projects to help the campaign. If you are a data scientist, a campaign needs you! Everyone needs to know which voters to target, and they’re hopefully looking for more ethical assistance than we have seen campaigns pursue in the past. Many campaigns can determine what kinds of voters they want to target, but they may lack the skills to find those voters within the mass of voter information available. Those who are great with analytics can help get data from the web sites to the voter analysis teams. Social media experts can help leverage Twitter, Facebook, Instagram, and other services to get messaging out effectively. Online advertising needs your expertise in marketing and advertising. Larger campaigns may need custom applications like HillaryBnB (an AirBnB-style app for canvassers).

Again, I had no experience with campaigns so these are just a few of the efforts I’ve observed recently, it’s a very non-comprehensive list of options. Each campaign’s needs are different, so I suggest checking with the campaign to see what is needed, rather than trying to offer specific projects.

Tackling the unknown

Though you are volunteering because you have expertise, sometimes what a campaign needs does not line up exactly. It’s a good thing we are an industry that is constantly learning! Lean on your existing expertise to get going.

I decided to help out a campaign with Google AdWords, as the lack of such a campaign was identified when I joined. Prior to June 1st, I had never used AdWords or done anything with advertising, online or otherwise. Yeah, it was intimidating. But I believe in my candidate, so I tackled it like any other tech I learned. I found some technical articles about how AdWords works and tips for novices, scrounged up some youtube video so I could see it, and then set up an account and got to work. After almost 3 weeks, I am starting to figure it out, and the campaign is benefiting from my efforts. Find something and dig in. You can grow your technical skills and help advocate for your politics at the same time!

While I will not pretend to be authoritative on AdWords, I want to share some things I learned, beyond the simple mechanics that you can learn through the documentation and tutorials:

  • Create the AdWords account with a central campaign account. Add additional administrators with campaign-specific emails, rather than personal emails. This makes it more difficult for someone to leave the campaign and take down advertising.
  • There’s a pretty decent iPhone app for AdWords, and it’s gives you some views not available on the web site, but you cannot edit very much on it. I am sure there is one for Android as well.
  • Google will provide a $100 coupon after spending your first $25. It will be sent in an email to the account owner’s email and it’s not automatic, you need to apply the code.
  • Google will also send an email advertising a free review of your account. I would wait a few weeks before calling, so that the specialists can see some data from your campaigns.
  • Impressions (someone seeing your ad) are free. Clicks (when someone actually clicks on them) are the only thing that costs you money.
  • Campaigns are made up of Ad Groups. Each ad group can advertise a different set of text and point to a different page, but funding is allocated on a Campaign level. You can add as many ad groups to a single campaign, but budget is allocated at the campaign level. You need to balance the number of campaigns and ad groups, and keep balancing them as voters’ interests change.
  • Each Campaign can also be targeted to different locations. You can limit some AdWords Campaigns to the political campaign’s region (by district for federal offices; by using zip codes for state and local offices) to focus your advertising, such as issues-based campaigns. Others may be made open to a wider area, maybe even the whole country, such as donation campaigns.
  • Each Ad Group can be made up of keywords which receive mostly-opaque scoring. The better the score, the better the placement. Keywords that do not result in Impressions receive a lower score and can drag down the score of the entire Ad Group. Disable keywords that do not work, or replace them with more specific and helpful keywords, to keep the scoring up. All the scoring is Google magic, and the effectiveness of this will vary quite a bit for every organization. Keep an eye on it.
  • Keywords are words or phrases that you want your ad matched to. You can also add Exclusions. Combine this with Search Terms results, which show the actual search someone used and the keyword or category it matched, to filter out Impressions/Clicks that are not helpful to your campaign. I have seen some really ridiculous search terms, including the amazing imagen you are an environmentalist giving a speech environmental due to population growth in the western united states that matched the keyword environment. Whoops, way too generic, it was replaced with a more specific phrase. Another was a search including the name of another candidate in another entire state and that one click ate the entire campaign budget for the day. Keeping up with Exclusions can save your campaign a lot of money.
  • Google watches monthly trends to determine when best to spend money. Your daily budget is better thought of as the average daily spend over a 30 day period. For example, Google may determine that you won’t get much out of ads on Saturday and spend close to $0, but the 2nd Wednesday of the month get the most impact and spend far more than your budget. You will only ever be charged 2x your specified budget, even if Google “spends” 2.3x your budget (I have never observed them going past 2.0x).
  • Almost everything you do with AdWords can be changed on the fly. However, there are two things to keep in mind:
    • Any new ad or keywords (and you cannot edit ads/keywords, you actually make a new one to replace the existing one) must be approved. It can take up to 24 hours to approve. You can add ads/keywords and disable them, then enable them when needed, to ensure they are approved prior to when they are needed.
    • Significant budget changes may flag a fraud alert. If you have a significant event-related campaign coming up, set it up at least 3 days in advance, as it can take 3 days to resolve suspected fraud. If you are spending $10 a day and want to increase that to $300 for a weekend event and your account gets flagged on Friday, it may be Monday before it is unfrozen and your event will be over.
  • You won’t find many AdWords tutorials that speak to political campaigns. It’s strongly associated with businesses. A few articles and charts mentioned Social Advocacy, which is probably the closest, but…
  • Success is difficult to measure. A business may track how many people click versus how many people order something. A donation campaign can track how many people donate, but an issues or awareness campaign cannot correlate visitors to the site with votes in a primary or general election. Conversion rates on their own won’t tell you much.
    • For many candidates, awareness itself is the goal. Many voters do not fill out the whole ballot and only check non-federal boxes if they recognize the name. Responsive Ads (as opposed to Text Ads) are fairly unobtrusive but can display small graphics. Logos are very helpful to start creating brand awareness.
    • Run at least two Ads for each campaign, much like you would do A/B testing at work. Review regularly and tweak the ads over time.
  • Advertising is not an island. Coordinate with the Social Media team and the event planners. If your candidate is going to be at a local festival or state fair in a few weeks, add some keywords for the event. If Social Media is blitzing on a policy, make sure common keywords will drive voters to your candidate. When things happen in the news that affect your voters, make sure their searches will bring them to your candidate. Likewise, you can review search terms people use and inform the other teams that these are some of the things voters are searching for and make sure the candidate speaks to those concerns.
    • This can feel very ghoulish or morbid. Much of news that drives people to politicians is going to be of a negative nature. We generally don’t call our elected officials when things are looking up. You WILL probably have to capitalize on an event that resulted in harm or death. In my case, unfortunately, there was a school shooting near the district. Ugh. But voters do want to know their candidate’s policies on subjects like school shootings. Be responsible and principled and above all, caring. Do not let the need to respond compromise your integrity or the candidate’s.
  • Your candidate’s party has other candidates running. Reach out to them for assistance, for ideas, for additional eyes on problems. You can get contact info from your county/state party’s offices, usually.
  • AdWords includes a large number of reports. Working in IT, our tendency is to encourage others to run reports themselves, but everyone on a political campaign is likely already spending all the time they can on their areas of expertise. It can be a huge help if you create/tweak reports for others, and schedule them to email the requester regularly.


Just because many of us make technology a huge part of our lives, we are not one dimensional. If you feel inspired by politics, don’t hide it, become active. I’ve discussed what you might expect if you join a political campaign, some of the work and expertise technologists can offer campaigns, and my experience in joining a campaign. Whether you contribute a few hours a month or hours every day, you will be a vital part of chosen campaign. That’s awesome! Participation in democracy is what makes our Republic so strong, and that of most democratic governments.

If you have any questions about volunteering, whether it’s technical or about the experience or something else, please reach out. You can drop a comment here, or @rnelson0 on twitter.

Enjoy, and thank you!

Opinion: Technology is always Political

I’m writing this opinion piece right now because of ongoing gross abuses of justice taking place in America right now. Don’t worry, as strongly as I feel about this subject, my blog will remain free of specific political advocacy (please follow me on twitter for my politics), but we absolutely need to talk about the relationship between technology and politics. I would love to see your comments here, or you can reach me on twitter if that is more comfortable.

As technical practitioners, we are often under a lot of pressure to focus on tech and minimize other subjects, especially contentious subjects like politics. These subjects can cause conflict and many of us are taught to avoid, rather than resolve, conflict. We are constantly told that talking about tech is great; maybe talk about your sports teams, music, craft beers – but never talk about politics. That’s divisive! Sure, sometimes politics may cause conflict and even alienate people, but that’s just an aspect of life, of who we are, and not something to box up and hide in a corner. We soon find that everything is political – and if it’s not, it will still be made political for us. Our politics reflect who we are, and we strive to grow and change over time, and it stands to reason that our politics grow and change with us. I find Scott Hanselman to be very eloquent on the relation of politics to self, probably because it comes up so often on his timeline:

Once you share your politics, like Scott, you may be told to stick to technology. But if everything is political, then technology must be political, too. It doesn’t exist in a vacuum. It never has and it never will. We must stop pretending that technology exists free from politics. Our community benefits from embracing this truth, not denying it.

Examination of our industry’s early history quickly shows the relationship between technology and politics. Let’s review the history of IBM during World War II. In 1933, IBM started cozying up to the Hitler regime to increase sales, starting with an innocuous sounding census. Having machine-tabulated census data allowed the German government to rapidly increase their prosecution of the Holocaust. Eventually, every Nazi concentration camp used IBM punch card technology to track prisoners – which IBM serviced under contract.

This was not an isolated act of the company or an oddity of IBM’s German subsidiary. This happened in America, too. IBM pursued and acquired the contract for the Japanese internment camps’ punch cards, at the same time its equipment was used for US Army and Navy cryptography. IBM was okay with the use of its punch card equipment to identify, round up, and track prisoners, even by a country at war with IBM’s home country, so as long as IBM got paid.

Neither of these efforts just “happened.” IBM employees developed the punch card technology. IBM employees had to contact the German and US governments to open sales channels. IBM employees  had to pursue and close sales contracts with the German and US governments. IBM employees had to provide support, spare parts, and even enter concentration camps to change the printer paper throughout World War II.

Numerous people were required to prosecute the Holocaust and the Japanese internment. Only a few were technologists, and fewer still worked for IBM. But all of them allowed their personal political and ethical views to be subsumed and harnessed to a political regime that enacted some of the worst atrocities in the history of humankind. We do not know exactly what these people intended, but history has recorded the outcome and judged it. No amount of, “Well, I didn’t mean that to happen,” or, “I didn’t want to take sides,” will ever change that.

“The only thing necessary for the triumph of evil is that good men should do nothing.” – Edmund Burke

And here we are again, in 2018, observing numerous authoritarian efforts, by both governments and public/private companies, to weaponize technology. Again, this does not just “happen.” Someone has to actively develop, sell, provide, and support these weaponized technologies. Each of us must inform ourselves of how the technologies we work on can be weaponized and used for evil. Implicit and subconscious bias will creep into products, but very frequently, efforts are consciously and overtly made to weaponize technologies. Original intent is never remembered, only the horrible outcomes.

Earlier, I stated that we should have and develop our own political views. In accepting that technology is intrinsically political, our use of technology may then become political advocacy. For example, If we are strongly in favor of a legal right, using a technology designed to interfere or suppress that right would be antithetical to our politics. Thus, our political views should drive our use of technology.

To avoid advocating for specific political views, I suggest that we all commit to a formal code of ethics that closely matches our political views. I highly recommend USENIX’s System Administrators’ Code of Ethics (more in Additional Links). It is straightforward and thorough, is compatible with most political viewpoints, and has stood the test of time within our industry. A chosen code of ethics is only helpful if we stick by it. Not just when it’s easy, but especially when it’s difficult!

So what happens if we do find ourselves involved with a system that can be weaponized, that violates our ethics and politics? How do we advocate our politics and maintain our ethical code when we have significant concerns? “It depends,” of course, on those concerns and how significant they are, on our relationship with the system, and on who we are providing for. We must each determine an inflection point where it passes from, “Can this still be saved?” to, “This cannot be saved.” Everyone’s inflection point will be different – we all have different politics and ethics, finances, health, and family situations, etc. – but we must each determine exactly where that point is for us.

Inflection point in hand, we can proceed to an action plan. We may be able to work fully above board, making cases to stakeholders and management. Or we may have to move below board, purposefully dragging our feet or working against the system. Maybe a change in vendors will address concerns or slow progress, maybe we just don’t do certain things at all. Find the appropriate monkey wrench that fits the gears of the system. We must plan our activities as we would any other technical work, laying out our goals and milestones and alternative plans for when disaster strikes.

We can also lean on each other. Reach out to your coworkers, your colleagues and peers, your friends and family, to voice your concerns and discuss remedies. We are not alone, and we can lean on or be the rock for each other. We may want to confide in just a few trusted people, to organize with our coworkers, or to become whistleblowers. There is so much nuance and possibility here that it is impossible to predict what actions will be required, but together we can determine what those actions are.

There may come a day when we find that we cannot stop the dangerous technology, when we have crossed that inflection point. We have to evaluate, honestly, whether there is still good we can do, or if we have truly passed the point of no return and need to walk away. Many of us will never be close to making these kinds of decisions, but some of us will. We must rely on our ethical codes and our planning to remain true, to see the point of no return, and to walk away, even if it costs us. To stay and actively participate knowing that we are now doing irrevocable harm would be costlier.

This is not a one-time deal. We must always keep our eyes open, evaluating our ever-changing political views against the work in front of us, applying our ethics to keep us on track, and making damn sure that we are never – metaphorically or literally – changing the printer paper in a concentration camp.

We will not build the software for concentration camps or to enable authoritarians. We will not spy on our neighbors or destroy democracies. We will commit to our ethical codes of conduct, and we will use technology to build the shining city on a hill.

Thank you.



Additional Links:

Burnout and Vacation Time

As the holidays and the accompanying vacation time inches closer, it’s a good reminder of the service vacation provides in avoiding burnout. Burnout is a serious problem in our industry and something we should all strive to avoid in ourselves and others. Of course, we rarely think we are burnt out until it’s too late. There are some common signs to look out for – are you constantly shuffling from project to project or thought to thought and unable to catch up? When you do sit down with a task, is it easy to become distracted, putting you even further behind? Are you feeling despondent or even depressed when work is on your mind, maybe even thinking about rage-quitting someday soon? You might be getting burned out. This tends to be pretty common for many of us when November rolls around, since we probably haven’t taken a good vacation since the last holiday season.

That makes the holidays a great time to take a vacation. You need it, and you probably are going to lose some or all of that time if you don’t use it now. If you haven’t scheduled it already, you should talk to your manager and get it scheduled now. Take time for yourself and back away from the edge of the burnout cliff. And the sooner you talk to your boss, the better chance you can actually take the time you want, since all of your coworkers probably want to take vacation at the same time and someone has to draw the short straw!

I suggest that you also work on ensuring that in 2017, you do a better job of planning your vacation throughout the year. One or two days here and there usually doesn’t do it, you need a long enough period of time that you can dump the weight on your shoulders and stand up tall for a while before you go back to work. Take a week off in April or June or September instead of hoarding vacation days. If you can, travel somewhere, but even a stay-cation helps a lot if it’s a whole week.

I also recommend touching a computer as little as possible. We’re nerds, we’re going to touch a computer at some point – and technically you probably can’t avoid it if you want to drive somewhere, watch some streaming movies, or order some food – but we benefit when we aren’t surrounding ourselves with the things that are causing us stress in the first place. Instead of using the vacation to catch up on all the new literature about Product X, pick up a good novel, watch a TV series you skipped when it was new, or maybe pick up an entirely new hobby. Build a chicken coop, or watch someone else build one! Anything that’s NOT work.

Above all, take care of yourself. Enjoy the whole year, not just the holiday season!

Non-conforming in IT

Many people in IT are non-conformists, compared to our non-IT brethren. Common themes include being good at math, enjoying games – video, but other tabletop games like D&D as well –  beginning our day with coffee or other forms of caffeine, can’t stand that sportsball, introverted, spend all day in front of a computer, and a few others. This certainly isn’t an exhaustive list, and not everyone hits these, but I imagine most reading this found a few things that describe them. Others, like coffee and caffeine, aren’t limited to IT people, but add to the collective list of commonalities. For a bunch of people going against the grain, we seem to be choosing a rather similar direction!

You don’t have to be a non-conformist in IT, though. I’m not! I do enjoy my video games and I’m on the introverted side of the spectrum. However, I don’t enjoy math beyond calculus and even prefer english and related studies, I don’t drink caffeine on a regular basis, I love team sports especially football, and my main hobby is woodworking in my garage. I’m the guy whining about the lack of just plain water at a conference or causing your eyes to glaze over when using football analogies in meetings.

If there’s any point to this short article, it’s a reminder that we all do really have different interests. It can be a little weird to be made the outsider of the outsider group, so be sure to celebrate those differences instead of using them against each other.

“No Deploy Friday” is a sign of IT Maturity

Last Friday, I made a pretty sarcastic joke about Friday deploys. Hopefully the image macro let you know it was a joke, though!

You shouldn’t be deploying on Fridays. Let’s qualify that a bit. “Deploy” in this instance means a high-risk change – a new product or equipment, a change in some protocol (moving from EIGRP to OSPF), or even some simple change that you’ve not done before so you don’t really know how risky it is – and “Friday “is your last day of the week. Patches on Friday? Up to you, though I don’t suggest doing so at 4:59PM. Is your shift 4×10, Monday through Thursday? Thursday is your Friday, get your deploys done by Wednesday. Is it a short week because of vacation plans? Maybe don’t deploy this week at all. Whatever the particulars are for you, “No Deploy Friday” is just the phrase for not deploying something brand new before you’re off for a few days.

Having the “No Deploy Friday” rule is a sign of IT maturity, for you and your company, in so many ways:

  • Realizing you cannot do everything. Sometimes you can’t do something when you want to. It’s better to accept that than try and force it.
  • Realizing that things go wrong. When you’re young, it’s easy to believe so fiercely that you know what you’re doing that you cannot accept that you might be wrong. With maturity comes the knowledge that even if you know what you’re doing, it can still turn out poorly.
  • Realizing that your decisions affect those around you. It’s important to recognize that when something does go wrong, you don’t exist in a vacuum. Other people are affected. Your boss and coworkers. Your customers. Your family. Your coworker’s family. You can make the decision that YOU do not mind staying late if something breaks, but you should not make the decision for your coworker’s spouse that they won’t be home for dinner.
  • Realizing that you matter. Work has to happen. But you don’t have to subsume yourself for it to happen. Your time off is yours. Friday movie night should be for watching movies, not fixing bad deploys. And instead of pushing a big deploy before you head on vacation, which pretty much ensures through Murphy’s Law that you won’t actually get to enjoy your vacation (or worse, you will but your company and users will suffer until you’re back on the grid!), talk to management and push the deploy out or find another owner. There’s also the more serious struggle with burnout that we all have. Reminder: You matter! If you need help, reach out to someone! We’ll listen!
  • Realizing that no matter what gets done today, there will always be more tomorrow! When I was younger, I sometimes thought I might “work myself out of a job”. What BS. If I ever actually did get everything done, it would have just given me time to find something else to do. For most of us, we’ll never actually empty out our task list. Since that’s the case, there’s no need to kill yourself trying to do so. This isn’t the same thing as saying, “feel free to slack.” It’s a fine line to know what NEEDS to be done today and what would really be nice to do today but would be more likely to be successful on another day. Knowing when you should postpone work until you have the time to do it right is another sign of maturity.
  • Realizing that “hurry” is the antithesis of “fast”. This one is counter-intuitive. We’re often told to, “move fast and break things.” I hate that saying. It really should be, “move fast and break things in development so you don’t break production.” That’s the real intent behind it. But even that’s not right. “Hurry” indicates your speed; “fast” is describing your velocity.  When you’re hurrying, it becomes easy to skip a step because someone hit you up on IM while you were working, or push through an error message because you want to leave to get to the movie on time. Those things end up costing you more time when you have to drop everything to troubleshoot or rollback. Now you’re hurrying at a high speed but have a negative velocity. You show maturity when you choose a velocity of zero over a negative velocity.
  • Realizing that Friday is great for “non-work” work. We work in a fast paced industry. We spend a lot of time working, but we have a lot of other things we have to do that we don’t always consider “work”. No-one likes doing their expense reports, but they’re great Friday work. We also have to keep up with new technologies and processes, and if you have an 8+ hour day with no deploys, it’s a good candidate for some contiguous learning time. There’s lots of ways to be productive while avoiding deploys, and your manager and finances will also be happy that they don’t have to hound you for your expense reports anymore!

It’s often a difficult struggle for maturity in IT. Just knowing what maturity looks like, doesn’t mean you can just go ahead and do it. If you’re in a culture of Friday deploys, you may have to lead the charge on this. If you’re the rogue IT person breaking the “No Deploy Friday” rules, talk to your coworkers and see what wisdom you can glean. Let maturity be your badge of pride, not your scars and war stories!

Getting started in IT: Years 0-4

Over the weekend, a really great hashtag came into existence, #FirstTechJob:


This in turn came from a great question about the requirements in job listings:


Please check out the hashtag, it’s a great sampling of the always humble, often mundane, beginnings of nearly everyone in IT. Some common themes were of course help desk support, managing printers and email systems, and managing or running ISPs (often including modems!). My, how times have changed. It did inspire me to talk a bit more about my journey in the hope that it may help some others on their own journey, whether they’re just getting started or have been at it for a while.

Getting Started


My very first professional job was working for a neighbor’s local PC business in the summer between high school and college. He sold then-high end computers (I think mostly 386s and sometimes 486s, but it’s been a while), with ISA graphics cards that took 30-60 minutes to render a single low-res frame, and needed assistance assembling them in a timely manner. The work itself wasn’t difficult – insert Tab A into Slot B, tighten Screw C – but I asked a lot of questions and learned a good bit about hardware. I made a few bucks, mostly spent at the local laser tag arcade, and most importantly was able to put “professional” work experience on my resume in addition to Wendy’s and KFC. Thank you, neighbor, for that first job!

After the first year of college, I lucked into a paid summer internship at a local engineering firm. The company’s owner was a friend from church and my Dad helped me get an interview – some of that was getting me in the door, but a good bit was getting me off my butt – and I was able to upsell my summer job and my schooling into experience. I did a number of responsibilities there over the next two summers. Migrating CAD files from an old unix terminal to the new NT 3.51 systems, then to NT 4.0; desktop support; network support; printers and plotters; and a bajillion other little things.

One memorable event was when Pittsburgh was struck by some severe weather (including tornados – a real rarity for that area!) and a lightning strike blew out the transformer outside our building. Always splurge for lightning suppression. Over half the hubs died and we got a fast track to switches. In 1997-98, that was ahead of the curve for many. There were of course many less memorable, but more important, things I learned. The most important was how to provide service and support to users and maintain a positive relationship. There were always trying people (I have actually seen someone stick a CD in a 5 1/4″ floppy and force the door shut, and it’s not pretty) but hey, I knew nothing about what they did, so why would I hold it against them for not being experts in my job area?

In spring of ’99, I was supposed to intern there again, but a hiring freeze changed that plan. I already had the college semester off and it was too late to schedule classes when I found out, so I canvassed and found two part-time jobs where I could maintain self-employment, my own schedule, and make money. I learned pretty quickly that I don’t want to be my own boss. That’s a lot of work, and sometimes I only had < 3 days of work in a week! I kept at this through most of ’99 and added “Y2K preparation” to my skillset. Note: you really want to retire before 2038.

In December of ’99, I found a full time job at a local IT consultant, except they weren’t local to me so I had to move. I am 99% certain the only reason I got the job was because I called the company every week asking if had openings and the owner decided it was easier to let me try a job on probation than to put me off anymore. Persistence pays off! This was my first full time, self-sustaining job. I stayed here for 3 years and did a little bit of everything: customer service was key to everything, large-scale OCR of court documents, web front-ends to said documents, Wireless WAN connectivity (pre 802.11b), and I really fell in love with network security.

Keep Going

That covers the first four years and a bit beyond, which gave me a really great foundation for the rest of my IT career. I would like to think I’ve done fairly well since then. These jobs may not seem like the kind of awe-inspiring jobs that everyone wants, but they were good jobs, with good people, and I appreciate how lucky I was to have them. I know it can be a struggle to get those first few jobs and years of experience, so if you can’t land a dream job out of the gate, know that you can find tons of other jobs that will benefit you and your career. IT is really diverse, and you may find something you didn’t know you were looking for; if not it will certainly help you with those “4+ years experience needed” jobs.

Good luck in your journey!

Root Cause Analysis: It’s Still Valid

You’ve probably heard it before: Root Cause Analysis (RCA) doesn’t exist, there’s always something under the root cause. Or, there’s no root cause, only contributing factors. This isn’t exactly untrue, of course. Rarely in our entire life will we find some cause and effect so simple that we can reduce a problematic effect to a single cause. Such arguments against RCA may be grounded in truth but smooth over the subtleties and complexities of the actual process of analysis. They also focus on the singular, though nothing in the phrase “Root Cause Analysis” actually implies the singular. Let’s take a look at how RCA works and analyze it for ourselves.

Root Cause Analysis is the analysis of the underlying causes related to an outage. We should emphasize that “causes” is plural. The primary goal is to differentiate the symptoms from the causes. This is a transformative and iterative process. You start with a symptom, such as the common “the internet is down!” In a series of analytical steps, you narrow it down as many times as needed. That progression may look like:

  • “DNS resolutions failed”
  • “DNS server bind72 failed to restart after the configuration was updated”
  • “A DNS configuration was changed but not verified and it made its way into production”
  • “Some nodes had two resolvers, one of which was bind72 and the other was the name of a decommissioned DNS node.”

Each iteration gets us closer to a root cause. We may identify multiple root causes – in this case, lack of config validation and bad settings on some nodes. Not only are these causes, root causes, but they are actionable.  Validation can be added to DNS configuration changes. Bad settings can be updated. Perhaps there’s even a cause underneath – WHY the nodes had bad settings – because RCA is an iterative process. We can also extrapolate upward to imagine what other problems could be prevented. DNS configurations surely aren’t the only configurations that need validated.

Multiple causes and findings doesn’t invalidate Root Cause Analysis, it only strengthens the case for it. If it makes it easier to share the concept, we can even call it Root Causes Analysis, to help others understand that we’re not looking for a singular cause. Regardless of what we call it, I believe it is absolutely vital that we continue such analysis, that we don’t throw away the practice because some people have focused on the singular. Be an advocate of proper RCA, of iterative analytical processes, and of identifying and addressing the multiple causes at hand.

A Full Stack What?

You’ve probably heard a lot of talk about the term “Full Stack Engineer”. You may even hear that everyone’s looking for one, so you probably want to be one to help your career. A Full Stack Engineer (hereafter FSE) is someone who doesn’t just know their one area deeply, but knows a bit about the rest of the stack. That depth of knowledge varies from very shallow to deep expertise, with the idea that the FSE knows how the different levels of the stack work together so they can make decisions that benefit the entire stack, rather than a local optimization that may harm the rest of the stack. You don’t want someone making an application decision that blows up the storage stack, or vice versa, so this kind of wide knowledge rather than deep knowledge is definitely helpful.

There’s a huge challenge to becoming an FSE, one of which is the sheer amount of layers in the stack to learn about these days. There’s so much to learn that it’s not actually feasible that any one person can learn all those layers deeply enough to really know the full stack. There’s absolutely nothing wrong with taking the journey toward Full Stack Engineer, but I think there’s another worthwhile goal out there:

A quipped about a “Full Stack Human,” a little bit of a tongue-in-cheek response to the overuse of the FSE term, but there’s some seriousness behind it. What it really means is that you should try and be a well-rounded person. In a sentence: Be more than a job.

A job is (hopefully) only 40 hours out of each 144 hour week and 2000 hours out of 8766 hours a year, less than 25% of what you do in a year. Sleeping should take up about 30% more – and it really should, we have to work very hard to not have a perpetual sleep deficit. Many of us will spend some of that remaining time trying to advance our work and careers, which is perfectly fine. This still leaves a lot of time, time in which we can find some hobbies and activities to enjoy so we’re more than just a working machine.

For exercise, I really like playing flag football. There’s a very diverse assortment of players out there and it’s far more entertaining than a treadmill or machine. When I feel creative, I enjoy woodworking. It requires deliberation, planning, and care in ways that my day job doesn’t – well, since I like having all of my fingers, anyway. I really like my sci-fi and fantasy novels, but I also make sure I fit some classics like War & Peace in between them. My wife and I don’t do anything truly adventurous, but we have been fortunate to visit a number of countries and enjoy their different cultures.

These activities gives you depth and adds dimensions to your character. (I realize I’m starting to sound like your parents did when you were filling out college applications, but bear with me a bit longer!) You meet other people and cultures and gain new viewpoints in which to perceive life. For example, in a decade of flag football, I’ve learned so many different ways to inspire teammates – and which ones don’t work! – and how to calm people down so they don’t lose the game.

I’d never get those experiences just by focusing on working my way up the stack at work, and those experiences help me out just as much at work. We talk a lot about encouraging diversity in tech, and in my opinion, it has to start in your personal life. A well-rounded person, a Full Stack Human, has those diverse experiences and can bring that diversity back into tech.

Your hobbies also give you a healthy escape from work. You aren’t just the project you released last week, and you shouldn’t kill yourself over work (figuratively or literally!). Identification and burnout can be a significant problem for everyone. If you don’t think so, you either aren’t there yet, or you’re there and you don’t know it! When you get too wrapped up in work – the deadlines are pressing down on you, politics got heated, you missed a family event because you were working late and didn’t even realize it – you need a safety valve to relieve that pressure and your personal time should help with that. PSA: If you’re struggling with burnout, please reach out to someone. We’re here to help!

Be a Full Stack Human. I guarantee it will be rewarding on its own, and it’s a huge step up on becoming a Full Stack Engineer!

2015 Recap: How did I do?

Just like I did at this time last year, it’s time to take a look at my goals for the previous year and see how I did.

Learn Ruby

I’d like to think I grok ruby at a more advanced level, now. I’ve written my first gem (and documented the ordeal) and contributed a number of patches to Ruby-ish projects here and there, mostly based around Puppet. I’ve also started writing “throw away” code in Ruby when possible, furthering my transition away from a bash-everywhere mentality. Grade: Pass

Blog more about Security

I started incorporating more security elements into my writing, but I haven’t really done a lot of security-focused writing. I only added one item to the Security category in 2015. I’m sitting on a bunch of drafts about security but am too timid to finish and publish them. Grade: Fail

Home Network

  1. I got my new home network up in running in the late spring, thanks to my partner-in-crime Mike SoRelle. I wrote an article about it as well.
  2. I made some progress here but not in the anticipated direction. I have all of my home network running Linux in Puppet and am working toward the same on the few Windows boxes. There was a lot of turmoil on the VMware side of things (5.5 updates, 6.0, 6.0 updates, changes to VCSA) and it slowed the work there. No IPAM, but I’m not feeling the burn very much because DNS at least is in Puppet.

Grade: Pass. But barely.

Expand PuppetInABox

I’ve learned a lot about software development in the past year. I’ve not only expanded and revamped PuppetInABox (support for Puppet 4 coming soon!), but I’m maintaining a few puppet modules, a puppet-related gem, and am actively participating in VoxPupuli (previously Puppet Community) and contributing features and fixes to Puppet itself. I think I’m making progress here, but still have a ways to go. Grade: Pass.

Propose a PuppetConf Talk

This was originally a goal to propose a VMworld talk, but I changed that as I didn’t have good subject matter for it before the CFP ended. I did submit a CFP to PuppetConf and was accepted! I presented in October and you can catch the video and slides online. I enjoyed the hell out of the conference and I dare say my talk did well, too! Grade: Pass.


I have made zero progress here. It was a busy year! I have until April to get this or renew the VCP and I’m not sure which it will be. Grade: Fail

Read War & Peace

This wasn’t on the list, but it was a personal goal. I’ve read a lot of Barnes and Nobles Classics and I love the Russian literature (Crime and Punishment in particular!), but at ~1100 pages of translated mid-1800 Russian, War & Peace was intimidating. I started this in the latter part of the year and I underestimated the time required to chew on it. I’m around 450 pages in after a few months. It’s been slow but very rewarding. You’ll be happy to know that in 1810, the Russians had meetings about having a meeting. What’s old is new again! Grade: C

I’d say I had a successful year. I didn’t hit all my goals, but like New Year’s Resolutions, I knew some goals would change and others wouldn’t be as important, it was more a guide for the year. I will be posting some new goals for 2016 shortly, though, so I can stay grounded this year as well!