The Constant of Variable Spend
In which Joe immediately caveats his assertions with an 'unless this doesn't apply to you' then goes on to say forecasting variable spend isn't that hard, and new music.
(Note to the reader: everything I wrote is generally true unless you work for a streaming company. I have no idea how the behavior of those environments works. Not an issue for most of my subscribed readers - for the most part, y’all work in legacy industries . . . and sales.)
The beauty of the cloud is its variable spend. The difficulty of the cloud is its variable spend. Variable spend gives you levers to manage your environment in real-time. Variable spend creates a tremendous risk of runaway expenses and blown budgets. It’s a good thing. It’s a bad thing. It’s variable! But is it variable, really?
My team's variable cloud spend forecast accuracy consistently beats other teams forecast accuracy of things like . . . depreciation, which is fixed. Perhaps it’s unfair for me, a guy from finance to come and dunk on engineers about the ability to perform financial exercises, but that doesn’t stop it from being enjoyable. The ability to more accurately predict variable costs than fixed costs does seem odd though, doesn’t it? Most of you might think that I must be lying because we all know cloud spend is completely unpredictable! That is the message, right?
Right?
Cloud spend isn’t really all that variable. It has the potential to be wild and unpredictable, but it’s not like cloud spend has a life of its own. If you have a centralized FinOps team, it’s not all that wild and variable.
If your cloud spend is variable, it’s because someone did something. Look at it from a micro-scale level. Launch a server, walk away, don’t touch it. That server has the same cost each day. Spend is steady. Then, you schedule it to shut off each night. Oh, look at that - the price went down! It changed. It changed cause you did something. You made it variable.
Or a more realistic example: You launched a server and left it running 24/7. Then you launched another server and left it running 24/7. Wowza - your costs just doubled! Cause you made them double.
The point I want to make is that at the level of estimated forecasts quite a bit of the variable spend is known activity. If it’s known, you can forecast it fairly easily.
So how to predict variable cloud spend? Three steps:
Understand your current run rate.
Do you know what drives the spend today? Yes - awesome. No? You don’t? Well, I’m not sure how you discover my blog, but I recommend you start looking at your cloud spend. If you don’t understand your current run rate of cloud spend, go get yourself smart and understand it. Go look at it. Point at things that do not make sense to you and say ‘what is this?’ “What is this?” is a very powerful question. Use it frequently. You’ll be amazed at what you learn.
What known projects/releases/programs are coming up in the period that you are forecasting?
New development up ahead? Get a high-level estimate from that team so you can include it in your forecast. The trick is having an ear in all the places work gets planned. I admit this can be harder than it sounds, especially in a large enterprise. Make friends with finance folks and portfolio managers, and even . . . bleh . . . architects. Flow of communication between these folks provides good info.
Are there any consumption reduction efforts in your future?
Most likely there are not. As Mike Fuller, head FinOps guy at Atlassian and who literally wrote the book on the topic, once said to me about cloud spend, “It goes up and to the right, doesn’t it?” A good way to think about potential reductions is if one of the items in Step 2 has an offsetting impact on something in your cloud space. Is something getting replaced? Are you moving to SaaS in any area? Another potential impact here is if - for some weird reason that remains unexplained and only makes sense in the context of your head - you have not done any reserved instances/savings plans/committed use discounts or any other sort of consumption-based discounting yet and are waiting to do so in the future?
Step 1 plus Step 2 less Step 3 = Your forecast.
Measure it against actuals. Investigate the differences to see why they happened? (Most likely that project that is going to add so much cloud spend is off to a slow start. Or they are learning how to use the new tool and expect their elevated run rate to not last.) Learn and get better.
What about autoscaling groups, Joe? HA! I got you and your oversimplified examples, you jerk! You can’t always predict everything! Everyone has an autoscaling group that goes out of control!
OK - now your brain is kicking in and you want to argue with me about how variable your spend is and why forecasting is not as easy as I make it out to be. Sure.
You have a server running that is part of an autoscaling group and all of a sudden due to increased use or bad scripting it starts scaling out. And it scales out. And it keeps scaling out. Oh crap, the spend is going up isn’t it? A spend anomaly! This is the variable spend that ruins our forecast!
If you have a centralized FinOps team, this is likely not a big deal. I’m just assuming you have some sort of tag strategy if you are reading this - (why would you be skipping to cloud spend forecast accuracy before you understand the value of tagging your resources) - so you will know who/what/where the resource is that is causing this variability. It is reasonable to expect a FinOps team leveraging tooling from either a third part of the native capabilities from the provider to see this activity in 48-72 hours. They can act and work with the team responsible to fix it. Yes, there is variability in this example, but it’s not uncontrolled. And if this takes you off your forecast? Well variability works both ways. You can act to try to reduce spend in non-production areas to get that budget back on track. That’s real-time decision making
And guess what folks, rare is it that any forecast (fixed or variable) is 100% accurate and goes according to plan over the course of a year. Regardless of what you are forecasting there is unplanned business activity. Events from the news cycle seep into your business and you have to respond. Things happen. When they do, it's important to be able to identify the impact.
I don’t think it’s all that much harder to forecast variable costs from fixed costs. It’s the same process. Perhaps the fear of variable spend forecasts is more about the real-time responsibility and holding teams accountable? FinOps is full-contact financial management. The advantage of variable costs is that if and when you go off forecast, you can do something about it. You can leverage that real-time decision-making to impact the spend immediately. You can’t do that with fixed costs. If fixed costs go out of control, financial leaders have to find some variable expense lever to pull to offset them . . . like labor.
Newish Music
Lately, I’ve been listening to a lot of music from the 70s. Lots of great music from that period but my kids have been starting to groan “Can’t we listen to something we know?” The answer is either No or I punish them by putting on Kidz Bop. However, there is a lot of music that I’ve really enjoyed that’s come out this year. Here is a selection:
Stop Making This Hurt - Bleachers
Solamente - Tai Verdes
Witchoo - Durand Jones & The Indications (Warning: This is gonna get stuck in your head.)
Summer’s Over - Jordana, TV Girl.
The Last Dance - St. Paul & The Broken Bones