29-09-2023
#personal
Summarise with AI
Summarise with AI
As long as I can remember, I have had an interesting relationship with sleep.
Be it the time that I fell asleep on a transatlantic flight with a blanket over me in such a way that my fellow aviators believed that I was a corpse hurtling towards San Diego that the cabin crew were neglecting to mention.
Or the time I fell asleep on the commode in Thailand after a particularly rough night of stomach disturbances to discover a cockroach the size of a large cockroach crawling up my leg.
I could go on.
However, despite being able to sleep through a hailstorm under a tin roof, I’ve never really figured out the best way for me to go about sleep to make myself more productive.
I’m always trying out new sleeping patterns and falling victim to social media influencers in T-shirts way too small for them touting the benefits of waking up at 2am to do yoga in an ice bath whilst shouting “CRYPTOCURRENCY” at the top of your lungs.
So I decided to take a data-driven approach to my sleep, combining data from my smartwatch with data from my productivity tracking tool to test whether there is a statistically significant relationship between how many hours I sleep and my productivity the next day.
The end result was… frustrating.
The correlation between sleep and productivity was a lousy -0.08 and even more, not even statistically insignificant at p = 0.46.
For those unfamiliar (sneaky way of saying those with a life) the correlation score measures the relationship between two variables. A positive correlation like 0.8 means that the more one variable increases, the other increases with it. A negative correlation like -0.8 means that as one variable increases, the other variable decreases.
The p value measures how likely this correlation is due to chance. For a scientific experiment, a correlation is considered statistically significant when the p value is below 0.05, meaning there is a less than 5% chance the correlation is due to chance.
So what does a nerd with too much time on their hands and hours sunk into a seemingly useless project do when faced with a setback? I blindly carried on until I discovered something vaguely noteworthy.
I grouped my sleep into distinct categories, for example 5-6 hours vs 6-7 hours, and saw some interesting results.
So clearly sleep does have some impact on productivity, but I was still after that elusive statistically significant result.
So using the skills I picked up from my statistics modules at university combined with my ability to unashamedly Google even the most simple of maths problems, I designed 6 additional hypotheses to test, with the intent to find out which sleep factors were significantly correlated with productivity.
These hypotheses, in no particular order because how would you even order these, were:
1) Overall sleep hours on weekdays affected my productivity.
2) Overall sleep hours on weekends affected my productivity.
3) Wake up time on weekdays affected my productivity.
4) Wake up time on weekends affected my productivity.
5) Bedtime on weekdays affected my productivity.
6) Bedtime on weekends affected my productivity.
The reason I split out weekends and weekdays like this is due to the fact "productivity" here only means my extracurricular coding and writing productivity, and ignoring my 9-5.
This means that factors like wakeup time and bedtime might have different impacts on productivity during the week as weekends I can roll out of bed whenever to code, whereas during the week it's limited to before 9am and after 5.30pm so it's best to test them separately.
Of those 6 additional hypotheses, only 2 showed a statistically significant correlation.
The first (and most statistically significant) was that waking up later on weekends is negatively correlated with productivity.
The graph above shows that the later I wake up on the weekend, productivity decreases and the earlier I wake up, productivity increases, with a p value of 0.03.
This may sounds completely intuitive and that's because, well it is.
But compare that with the fact that no similar effect was shown for waking up earlier during the week, and you start to see why it's interesting.
In fact, during the week, the correlation between wake-up time and productivity was a positive one at 0.18, meaning that the later I wake up the more I get done, however this wasn't statistically significant at a p value of 0.16.
The second (and more statistically tenuous) correlation was that, despite all logic and reason, somehow going to bed later on weekdays leads to higher productivity the next day, with a correlation of 0.25 and a p value of 0.05.
Now before the nerds revolt and I’m pelted to death with Inhalers (I use one so I am allowed to make this joke), technically, this means it’s insignificant as the p value has to be under 0.05 rather than exactly 0.05.
However, some tests use p<0.10, so I decided to include it in this blog as it’s a pretty bloody interesting result, this isn’t a dissertation and it’s my website and I can do whatever I want here.
This really threw me for a loop. How could going to bed later (and getting less sleep) possibly make me more productive the next day?
Then it hit me.
I work a fixed hybrid working schedule, meaning every Monday and Thursday I’m in the office.
Potentially, this correlation is actually the byproduct of my WFH days, where I go to sleep later the night before as I don’t have to commute, and am more productive as I have more energy and time to work on my side projects before and after work.
So I tested two more hypotheses because I have a serious problem:
1) The time I go to bed before I have to commute affects my productivity.
2) The time I go to bed before I WFH affects my productivity.
And this caused me to learn the hard way that science is fun because it's unpredictable...
Somehow, the stronger and more statistically significant relationship was between bedtimes on nights before I had to commute and productivity the next day, and it was a POSITIVE ONE at 0.40 and a p value of 0.05.
The correlation with productivity was weaker for bedtimes before days I worked from home was 0.15 and the p value was greater at 0.35.
This means that the data suggests the later I go to bed before days where I have a commute, the more productive I am that day.
I absolutely cannot for the life of me work out why this occurs so if you have any inkling, please let me know by leaving a comment on my feedback forum, The GC.
For the meantime, all I can do is leave you with a quote from one of the greatest movies of all time.
There is no direct linear correlation between my overall sleep hours and productivity, however it appears that between 5 to 7 hours is my sweet spot.
There is a direct and statistically significant relationship between waking up earlier on a weekend and my productivity on that day which isn't replicated during the week. From this point on, I’m going to try and make sure I am up before 8am on weekends and am going to be more relaxed about my wakeup times during the week.
There is a tenuous positive relationship between my sleep times on weeknights (especially nights before I have to commute) and my productivity the next day, however the exact mechanism of why this is needs to be looked into further.
This is based on three months of sleep data, so it isn’t the largest sample size.
My watch probably doesn’t have 100% accuracy in tracking actual sleep duration.
I am using productivity in a very niche field as a basis for “success”, when life consists of much much more than just being productive.
Purely tracking time for both sleep and work isn’t great as it doesn’t take into account quality of either i.e. working for 20 minutes can sometimes be more impactful than working for an hour.
Correlation doesn't imply causation yada yada yada we all know this bit
Load comments
Comments
No comments yet, be the first!