In this marketing tech in transition series, my team and I highlight four imperatives: Big Data, Cloud, Identity Resolution and Artificial Intelligence. and share how applying these imperatives will transform your marketing tech.
It's late in the year 2000. For almost two years we've been working with a client to help them design architecture to support their marketing automation and analytical needs. At the core of the architecture was a traditional RDBMS sitting on top of a massively parallel machine. We calculated the total cost of ownership over five years, as well as ten years. We calculated the ROI. In about seven months they were up and running. That was 2001. The massively parallel machine would cost that client over two million dollars to just stand it up. You can guess how much the RDBMS cost to license on top of that architecture. This was a huge investment and commitment. Fast forward and in less than two years, that same architecture would drop in price over 40% when we procured it for other clients and the cost would continue fall off a cliff every year thereafter. Today we can provision data architecture with vastly more processing power, coupled with an array of analytic and marketing tools, in less than two hours (really minutes if I don't count all of our validation and verification). I can pay for that architecture on my credit card, if I so desire. Adapting to the cloud makes this possible.
Our adaptation to the cloud started with a challenge to the team. “Take our process of procurement of servers, 3rd party and proprietary software installs, configurations, and validations of our marketing automation environment and shrink that to a day.” This was a process that normally took three months when the challenge was issued in 2010. As stated above, it doesn’t take a day, but only minutes now. The main reason for issuing the challenge was to shrink time to market, but along the way we discovered other important benefits as well. Operational overhead became virtually non-existent. Our risks decreased and consistency in quality increased. Configuration standards become immediately memorialized.
When we set up an environment terrestrially, there is a lot of paperwork, tickets, documentation, etc. for each architecture configuration. Hand-offs increase change management risk. However, our cloud deployment is the same every time and it’s automated.
We didn’t just want to push what we were already doing to the cloud, though. We wanted to build an interface on top of the cloud to allow our architects and developers to make changes and provision environments. We began with Amazon Web Services (AWS) and created our “SkyNet” user interface (a bit of a tongue-in-cheek reference to Terminator), based on an API layer that calls Ansible playbooks and some direct calls to AWS where it made sense. Ansible playbooks are just a set of things you want done. You can string playbooks together. You can pass output from one to another. We broke down the entire procurement of an environment into a micro service orientation.
A lot of our developers had never used AWS. We needed to pick a framework. Puppet, Chef and others were too complicated and oriented more toward admins. Ansible, an orchestration layer built on python, was really developer friendly. In a week, we were able to create a basic playbook and server automation. Also, Ansible was agentless, which means we didn’t have to install any software on the servers we’re going to manage which simplifies the implementation in comparison to other tools.
Now databases, app servers, antivirus, monitoring, logging, etc. are all automated and can be auto scaled. Auto scaling allows us to spin up an exact copy of whatever we need, when we need it. If we set CPU/Memory thresholds, there will be a Cloudwatch alarm, and that alarm is set to trigger a scaling event. The auto scaling group has specification and configuration for the type of server or servers needed. Now when our clients have a promotional event or a spike in their business, the system immediately can adapt and scale.
For any marketing technology architect, understanding how to leverage the cloud is an imperative. We not only get an overwhelming speed-to-market advantage, but we have more reliability, consistency and quality in our deployments. We have greater automation, reducing our overhead and we have adaptability you just couldn’t get easily any other way. Leveraging the cloud changes the game in many ways. You’d be a fool to not have a strategic initiative in place to align cloud deployment to your marketing technology architecture.
Tomorrow, I sit down with the Conversant team to discuss identity resolution.