Tonight at 8pm the Nanodegree course I enrolled on, Predictive Analytics for Business, finally opens. You would think I would be writing about that tonight, but no. Even though I am pretty excited (and a little intimidated) by it, I have something else in mind.
Over the last couple of weeks I have spent a good amount of time working with PowerQuery to put together daily figures into a weekly, monthly and annual view. It all works, if the data is there. And that’s the problem.
In business, not all the data you want will be there, and if it is, it often won’t be in the format you want it to be.
So I imported all of the flawed data into one workbook and tried to do my cleaning and processing in the same place.
It took aaaaagggggeees loading every time I went to view the PowerQuery editor. And after it had finished with the cleansing step I thought I needed, I would discover another one I had to do.
This happened numerous times, I’d say more than 10. Each time was a different thing that I could potentially have foreseen, but I was against the clock. Foresight and planning don’t tend to play nicely with deadlines.
Again and again I made fixes, tested the result and came up lacking. So much of what I needed to be done was done, but that remaining 10% that I thought I could knock out in a couple hours? No way.
So at half 2 today, having worked for over 7 hours on this thing, my manager advised me to throw in the towel and take a break.
I had failed again, at work this time instead of in studying.
After my break, I sat down with my failed work, copied across the pertinent data and went back to the old model which doesn’t have daily breakdowns in it. And it worked, because the data was there, just not split how it needed to be for the newer, fancier model.
Getting everything together for the old model took 40 minutes. Trying to make the new model took over 7 hours.
When you’re in the middle of something and you can see the light at the end of the tunnel, it’s difficult to just stop and look at the path to the end objectively. I kept thinking that if I overcame just one more hurdle, I would be there.
Really, I saw the warning signs early on. My new model was unfinished and so were the data inputs, but I didn’t take notice of those red flags, I thought I could overcome then. Until I didn’t.
There is some reward in giving in, I got to see how understanding my colleagues were in such a situation. That was worth at least some of the struggle.
And tomorrow is another day, and I know what went wrong.
Next time I’m going to ensure that the easy option remains the easy option, and follow that one through sooner.
Maybe this isn’t the “gritty” way, but there is value in knowing when to throw in the towel, and I hope to learn that properly some day soon.