Etsy CTO On Its Conservatively Crafty Tech Philosophy

Etsy CTO Kellan Elliott-McCrea sat down with TechCrunch recently and talked about the Etsy process when it comes to selecting technology, and it is very much a company that has clearly defined methodologies and approaches for everything it does. In his words, to keep the business going with a relatively small staff, “we need to consistently punch above our weight.”

The online artisans’ marketplace sells some unique products and has some unusual needs when it comes to IT. The company sets itself apart in a number of ways. Instead of using cloud services like many young organizations, it mostly shuns them preferring to bring as much as possible in-house.

Etsy, which recently had a hugely successful IPO, is remarkably conservative in some ways, yet always looking to learn. For the most part it prefers to deal with a handful of technologies it can master and understand at a level of expertise few companies can attain. These include PHP, JavaScript, Java Scalar and Hadoop.

The company has 685 employees of which approximately a third are engineers. It boasts a community of 1.4M sellers, listing 29M items with 19.8M active buyers. Total gross sales in 2014 were a whopping $1.93B.

According to numbers supplied by the company, it updates its code as often as every 20 minutes, and as many as 70 times per day (with 29.40 on average). Last year it launched 10,000 deploys. In short, it achieves an amazing level of productivity.

Method To Its Madness

Etsy wants to have as much control as possible over every aspect of the technology stack. While it doesn’t make its own hardware as Facebook does — Elliott-McCrea admits he’s jealous of that level of customization — it carefully analyzes each technology decision.

It all begins with the idea that the company wants to be a learning organization. He recognizes that the technology world is constantly in flux and he doesn’t want to adhere rigidly to a system. That means the company process involves iterating, measuring, learning, investigating and experimenting in a continual cycle of learning and development.

“Because we are Etsy and because we believe in learning and iteration and the human-centric approach, it’s a process rather than a single point in time. We have a new technology process that we go through. We start off with a lot of clarity and we are going to use the simplest possible thing and the technology we have the most confidence in,” he told TechCrunch.

It’s Always Sunny At Etsy

Part of that approach is actually eschewing the cloud, at least when it comes to its engineering, operations and development efforts — Email and document sharing is another matter we will get to shortly.

The company owns and operates its hardware and networks in its own datacenter. Part of the reason for avoiding the cloud is control, but not necessarily just from a security perspective as you might think. The real reason is Etsy personnel want to get under the hood and understand every aspect of every piece of technology the company controls.

Etsy once used AWS Elastic Map Reduce for Hadoop (from Amazon Web Services), which helps with processing large amounts of data. For many companies, using a service that took away much of the underlying complexity would be a positive, but for Etsy it was the opposite. It wanted to know how Hadoop worked, and the only way to do that was to bring it in-house and figure it out.

“As [Hadoop] became a more important part of what we do, we looked at a handful of metrics around utilization in terms of how much it was costing us, if we were getting great access to it, was a large percentage of the company using that resource and did we have insight into how well it was working,” Elliott-McCrea said.

When he looked at it through this prism, he realized it made more sense to bring Hadoop in-house where his team could understand every aspect of the tool. “We were able to test and experiment and operationalize and work with our DevOps culture with [operations] and engineering working together where it made more sense to bring it in-house,” he explained.

And he found it worked with a 10x increase in terms of Elastic Map Reduce usage. This has had a significant impact on the company as data has played an increasingly important role in solving some of the organization’s toughest problems.

“It’s perhaps not obvious when you look at Etsy, but things like relevance and ranking and recommendation are particularly hard in a marketplace like ours.” Bringing Hadoop in-house helped them learn how to get the most from the tools.

Keeping It Lean

One way Etsy drives this kind of commitment to deep understanding that seems to underlie its technology choices, is limiting the number of tools it uses as a company. It would rather be the master of few tools than have a broad knowledge across many different ones.

“We want to give people the time and space to really go deep and to understand these tools. That means being able to pop open the hood and look inside and make changes and not run into a wall where you can’t fix a bug in the kernel or database.”

“We want to give people the time and space to really go deep and to understand these tools. That means being able to pop open the hood and look inside and make changes and not run into a wall where you can’t fix a bug in the kernel or database.”

“We are a strong PHP shop. We used to use a wide variety of technologies, now we use a handful. We use PHP, JavaScript, Java Scalar, but we went from a large number of languages to a small number of languages — and we are really good at PHP,” he explained.

One thing this level of expertise has allowed them to do is to partner with Facebook (another big PHP shop) to help transfer a significant piece of its internal infrastructure to the Hip Hop Virtual Machine (HHVM), which improves performance and speeds up coding and development when using PHP.

Elliott-McCrea reports that his company is now a major contributor to HHVM because the expertise it has built up allowed it to be fully involved in its development.

“We don’t treat technology as something off the rack. It’s about mastering your tools and that’s something that’s a recurring theme, and again mastering your tools, improving them and then giving back,” he said.

To that end, Etsy has built and contributed many tools to open source (much like Netflix, Facebook and Google). This not only allows others to take advantage of those tools, but also potentially to improve upon them. In fact, Etsy has four pages of tools it has contributed to open source on Github.

Shadow IT Not Welcomed Here

Shadow IT, that idea that employees can and will provision their own software (because they can), is everywhere because IT is powerless to stop it. Elliott-McCrea, however, is adamant that his company has no patience for freelancing when it comes to software and he claims it’s not happening.

In a company with over 600 employees, it’s hard to know what they’re all doing in this day and age, but Elliott-McCrea believes his security team’s low-key, yet thorough, approach combined with the overall company philosophy keeps everyone in line.

“We do have a policy around [Shadow IT]. Again it’s not so much policy as process and evocative questions. No, you can’t just use tools you want. The way we get great is using a small number of tools and we add new tools to the stack cautiously.”

We do have a policy around [Shadow IT]. Again it’s not so much policy as process and evocative questions. No, you can’t just use tools you want. The way we get great is using a small number of tools and we add new tools to the stack cautiously.”

Still, he insists, theirs is not a culture of ‘no.’ As you might expect, there is a process for introducing new tools and employees have to convince Elliott-McCrea why it’s necessary to add a new tool, and why it’s better than the ones that have been carefully added to their tool box. He says that policy is more relaxed for developers than for the production environment. He and his team are quite strict about what employees can put on production and the internal network.

Going Mobile

Mobile is a big deal at Etsy — accounting for half of its traffic. It offers iOS and Android apps, and even has an Apple Watch app. In-house it’s a different story though. As with its other approaches, Etsy tends to be more cautious about allowing access to the company network from mobile devices.

Interestingly, employees can use Google Apps For Work from any device. That means employees have access to email and work documents via Google, but it’s rare for employees to get remote access to any other parts of the system outside of the office.

“We have a mobile VPN and it’s on a case-by-case basis right now. I wouldn’t say we have a solid answer [around mobile access], but it is evolving,” he said. The issues around allowing access on the net

work from mobile devices is complicated by the international nature of the business. Etsy has offices in Berlin, London, Dublin, Paris, Melbourne and Toronto.

Interestingly, much like Facebook, Etsy began developing apps for mobile devices by creating separate development teams for mobile and web. It quickly learned that this approach couldn’t keep up with the scale required by a company like Etsy. That’s when it decided to distribute mobile responsibility across the team instead of isolating it — the same conclusion Facebook came to with their development teams.

Etsy clearly has a method to make the operation run as efficiently as possible, but it can’t operate so rigidly in service to that efficiency that it could miss tools that would improve the company’s systems.

“One thing that’s inevitable is change. We haven’t figure out every solution. [Sometimes] the things we think are true will be wrong,” Elliott-McCrea said. And that’s why Etsy builds learning and iterating into every technology decision it makes.

Featured Image: Philip Tellis/Flickr UNDER A CC BY-SA 2.0 LICENSE

from TechCrunch http://feedproxy.google.com/~r/Techcrunch/~3/gC73fWzw2Yc/
via IFTTT