Is this a real problem, or just an anomaly?

One simple way to determine if an event was significant or just noise, is to do some basic anomaly detection. This works by finding the standard deviation and mean of the data prior to the event, then seeing how many standard deviations out from the mean the value is. In using this statistical technique instead of guessing or eyeballing the data gives you a reliable, consistent method for determining how important a deviation a worrisome new data point is. You find the upper bound for anomalies by adding 1x, 2x, or 3x the standard deviation to the mean, and taking it away to find the lower bound. If it’s between those values, it’s an anomaly.

This is key if you want to avoid continuously chasing your tail as an analyst, because it can tell you when to dive in to solve a problem, and when it makes sense to just relax and wait for more data. It's likely that if something isn't a true anomaly it will revert back to the mean given a few more days or weeks. This technique is closely related to quartiles, for example dividing the data into 4 buckets using the QUARTILE function in GSheets, you’ll see the difference between the 3rd and 1st quartile, is approximately 1.35 times the standard deviation.