Tag: forecasting

  • The Number Was Wrong by 2x, and I Found It by Predicting the Wrong Number in Advance

    The forecast had been “very inflated” for weeks. Nobody could say by how much, or why — just that the projected end-of-day numbers didn’t match what actually happened, often enough that people had started mentally discounting them.

    I went looking for the model first, because that’s where you look. What I found instead was that the fifteen-minute data feeding the forecast was cumulative — a running total for the day, not a fresh count per interval. I confirmed it the boring way: pulled the raw rows and watched them climb, strictly, all day — never dropping, only ever adding on top of the last number. That’s the signature of a running total, not a series of separate readings.

    The actual bug wasn’t in that data. It was one step downstream, in the code that consumed it. Somewhere in the pipeline, someone had written a loop that summed those cumulative numbers as if they were fresh increments — adding a running total to another running total to another, compounding a small mistake into a large one. In one function I found the two clearest evidence of it happening live: one number in the loop correctly took the maximum value across the period, and the number right next to it — same loop, same author, same line count away — used addition instead. One field right. One wrong. Nobody had noticed, because both numbers looked plausible in isolation.

    Before I told anyone what I’d found, I wrote down what the bug should produce if I was right — a specific projected number, checkable against the live dashboard within the hour. I did the math, then went and looked. The dashboard read almost exactly what I’d predicted it would if the bug was real: roughly double the actual count, growing toward quadruple by late afternoon as more cumulative snapshots piled onto the sum.

    The struggle wasn’t finding the bug. It was resisting the urge to declare victory the moment I found *a* plausible cause, instead of confirming it actually explained the whole shape of the problem — the way it got worse through the day, not just that it was wrong. A bug that only explains part of a symptom isn’t the bug yet.

    The lesson: if you can predict a specific number a bug should produce, and go check it against reality before you tell anyone you found the answer, you’ve turned a guess into a proof. That one habit is the difference between “I think I found it” and “I found it.”

    Next: fixing the aggregation at its actual source, not patching the number that comes out the other end.

  • Two Forecasting Systems, and Only One of Them Was Real

    The forecast was wrong, and everyone knew it was wrong, and nobody knew why.

    That’s the specific, uncomfortable place to start a debugging session from. Not “there’s a bug” — there’s a number, printed on a dashboard people actually look at, and it’s been quietly too high for weeks. Not broken enough to alarm anyone. Wrong enough that nobody trusted it.

    I went looking for the model. That’s the first mistake, and I want to be honest about it: I assumed there was one forecasting system, and it had a bug in it. What I actually found, once I started reading the code instead of the documentation about the code, was two separate systems living in the same codebase. One was a machine learning model — trained, evaluated, and then never actually saved anywhere. A path that looked live in the architecture diagram and had been dead for who knows how long. The other was a much simpler statistical system, tracking completion ratios against historical patterns, retrained regularly, and — when I actually tested it in isolation — producing numbers that looked correct.

    That was the surprise. The model everyone assumed was doing the forecasting wasn’t running at all. The real system was quietly fine. Which meant the inflated number wasn’t coming from a broken forecast. It was coming from something downstream of a correct one.

    The struggle wasn’t finding the second system — it was sitting with the discomfort of “the thing I was sure was broken turned out to be working,” and having to admit that meant the actual bug was somewhere I hadn’t looked yet. It’s a specific kind of frustrating to disprove your leading theory a week into hunting for something. The instinct is to keep pushing on the theory because you’ve already invested in it. I had to let it go and start over from “okay, if the inputs are right, where does the number actually go wrong.”

    The lead I ended up with, and haven’t fully closed yet: the projection math likely divides a current count by a ratio measured at a specific hour, and if that ratio is underestimated early in a shift, the division inflates everything downstream of it — a small early error compounding into a big late one. I don’t have it fully proven yet. But it’s a real, specific, testable hypothesis, which is further than “the forecast is wrong” ever got anyone.

    The transferable part isn’t the bug. It’s that “which system is actually running” is a question worth asking before “what’s wrong with the system,” every time — because the two questions send you down completely different paths, and only one of them is real.

    Next: instrumenting the actual division step directly, hour by hour, instead of trusting the summary numbers on either end of it.

  • How to Forecast End-of-Day Call Center Performance

    By mid-afternoon, you can know where your floor will close by end of day — accurately enough to make the remaining hours a decision, not a guess. Here’s how intraday performance forecasting works and what it takes to build it.

    ## The Problem With Yesterday’s Numbers

    Most contact centers have end-of-day metrics. Dials, connects, conversion rate against target. Those numbers are accurate, useful for trend analysis, and arrive the next morning.

    By the time you see them, the day is already over.

    The decisions that drive outcomes happen during the day — in real time, when hours remain to influence the result. Do you push harder in the final stretch? Adjust campaign priority? Pull a server that’s underperforming? Those decisions get made in the afternoon with one question underneath all of them: where are we going to close?

    If you’re answering that question with yesterday’s data and experienced intuition, you’re working with an information deficit that compounds every day it stays open.

    ## How Intraday Forecasting Works

    The system records dial conversion rates at regular intervals throughout the business day. Not a snapshot at end of day. A continuous read of how the floor is performing as it performs.

    Every morning, before the floor opens, the model retrains. It processes the intraday conversion patterns from previous days — how conversion tends to develop through the morning, when it typically accelerates, when it softens, how afternoon performance differs from morning — and calibrates to the current operation’s historical data.

    As the day runs, the forecast updates on a regular schedule. Each update incorporates actual conversion data that’s come in, narrowing the prediction window.

    By mid-afternoon, with hours remaining, the model’s error range has compressed enough that the closing metric is predictable within an actionable range. Not a rough estimate. A forecast with a documented accuracy track.

    **What this changes in practice:**

    Before the forecasting system, the afternoon conversation was backward-looking: here’s where we are, here’s where we were yesterday, here’s the gap. The decision about the next few hours was judgment — experienced judgment, but judgment without a forward projection.

    After the forecasting system, the afternoon conversation is forward-looking: here’s where we are, here’s where we’re going to close, here’s what the remaining hours need to produce to change that number. The judgment still applies. But it’s informed by a projection that’s been validated against actual outcomes rather than intuition alone.

    That’s a different kind of management posture. You’re not reacting to what happened — you’re positioned in front of what’s about to happen.

    ## The Complication: Point Estimates Break When You Need Them Most

    The obvious version of intraday forecasting — average dial conversion rate over recent days, extended to end of day — works in normal conditions and breaks exactly when conditions are abnormal.

    Days after holidays follow different patterns than regular days. Days with agent attrition don’t produce the same intraday curve as fully-staffed days. A mid-day list quality shift — new inventory loading into an active campaign in the afternoon — changes the conversion trajectory in ways a backward-looking average can’t capture.

    A point estimate that’s wrong on the days that matter most is useless. Operations managers stop trusting a forecast that fails them when they need it, which is precisely when conditions are unusual.

    **The correct implementation produces a confidence interval, not just a point estimate.**

    A point estimate says: we’ll close at X conversions.

    A confidence interval says: we’ll close between X and Y conversions, based on current trajectory and historical variance.

    The confidence interval tells you two things the point estimate doesn’t. First, it tells you when the day is trending outside normal bounds — when the interval is wider than usual, the model is operating in territory with less historical precedent, which is a signal to pay attention. Second, it tells you when a result is well-constrained versus genuinely uncertain — a narrow interval with hours remaining means the outcome is largely determined; a wide interval means the remaining time is more variable than usual.

    **Additional inputs that improve accuracy:**

    Day-of-week weighting. Each day’s patterns should be compared to its own history, not the full week averaged together. The model needs to know what kind of day it is.

    Agent count as a real-time input. Utilization affects conversion. A floor running below normal staffing has a different conversion profile than a full floor, and the model needs to account for that explicitly rather than absorbing it as noise.

    ## What Building This Requires

    **A data collection layer.** Conversion rates pulled from your dialer API at regular intervals throughout the day, stored with timestamps. Convoso’s API supports this; a scheduled script on a consistent cadence collects the signal reliably.

    **A retraining pipeline.** Each morning, before the floor opens, the model fits to historical intraday patterns. The features that matter most: time of day, day of week, agent count, and recent conversion velocity. A well-structured ML regression model with the right features outperforms a complex one with the wrong ones — the goal is a forecast accurate enough to act on, not a showcase of modeling sophistication.

    **Validation tracking.** The model’s predicted closing range logged against actual outcomes, daily. This is how you know when the model is drifting and needs recalibration, and it’s how you build the track record that makes the forecast worth trusting. A model you’ve been validating for several months has a credibility that a newly deployed model doesn’t.

    **A delivery layer.** The forecast needs to appear where ops managers look — a dashboard, a scheduled Slack message, a Google Sheets integration. A forecast nobody sees is the same as no forecast.

    ## Frequently Asked Questions

    **How accurate is the forecast by mid-afternoon?**
    Accurate enough to make the remaining hours actionable rather than reactive. The exact precision depends on your operation’s historical variability — a stable floor with consistent staffing and consistent list quality will have tighter forecast intervals than one with high day-to-day variance. The value isn’t in the precision of the number — it’s in the direction and whether you’re trending toward or away from target.

    **Does this require machine learning expertise to build?**
    The approach is an ML regression model with appropriate feature engineering — time of day, day of week, recent velocity, agent utilization. The complexity is in getting the features right and building the validation discipline, not in the modeling approach itself.

    **What data do I need to start?**
    Historical intraday data at the interval you want to forecast at — ideally several months of regularly sampled conversion rates. If that data isn’t archived, starting with collection now and building the model after accumulating enough history is the right sequencing.

    **Can this work if our staffing varies a lot day to day?**
    Yes, but agent count needs to be an explicit input to the model rather than something it absorbs implicitly. Days with unusual staffing should be labeled as such in the training data so the model can account for the pattern, rather than treating them as noise.

    ## If You’d Rather Have This Built

    I build intraday performance forecasting for contact centers. If you want the data collection, the retraining pipeline, and a confidence-interval forecast that surfaces where your floor will close before the last hours are gone — start here: rfditservices.com/intake.html

    The first conversation is free.