The document discusses an alternative approach to scientific inquiry based on tracking unexpected or outlier observations from self-experiments rather than relying solely on rigorously designed experiments. It involves logging unusual events and recurrent experiences outside normal expectations to identify associations between interventions and outcomes. While prone to false positives due to lack of controls, this approach can provide insights more quickly and capture richness from extreme observations that may be lost with traditional measures. It illustrates some examples where outlier observations helped uncover meaningful associations.

1. How much can one data point inform?

2.  The scientific method most of us will be
familiar with.
 Aims to maximize accuracy
 Make informed hypothesis
 Design experiment to test
 Analyse and adjust if necessary

3.  Novel experiences besides those intended to
be measured.
 Concordant
◦ Weight loss
◦ Less mental fog
 Discordant
◦ Improved productivity
◦ Feeling sleepy earlier
◦ Jaw tension

5.  An observation that lies outside normal
experience.
 Generally inadequately accounted for by
conventional measures of significance.
 ‘One perfect day’
 ‘Turkey problem’

6.  A single outlier generally only indicates
‘weak’ associations.
 Prone to false positives but useful as starting
point.
 Zero frequency
 Convergent events

7.  Hereditary condition causing painful cramps
and gastrointestinal upset.
 Irregular av.~1/month in frequency
 Last experienced 04/03/12
Polyphasic sleep
Diet overhaul
Cold thermogenesis
Unknown
2012

8.  Device measuring HRV.
 Subjective feeling of being ‘calmer’ and more
present.
 Two outliers: felt fine, but could not get
device to turn green.
 Next day spent fixing/scraping code due to
glaring errors.

9.  May-12 development of a bad rash in the
small of the back.
 Tried balm, extra padding, change chair, etc
 Rash went away.
 Jun-13 Shortly after moving workstations,
had relapse.
 Change of chair as common element.

10.  Track events outside normal experience and
recurrent, unusual events.
 Conduct self-experiments as normal.
 Build associations between experiments and
observations.
 Strength of association determined by degree
of changed experience.

11.  Very little effort spent to log.
 Does not require a priori hypothesis.
 Less time needed for results.
 Ability to find meaning in fairly trivial or
transient events.
 Visceral rather than abstract associations.
 Misses slight incremental improvements.
 Lacks rigour

13.  Significant outliers

 Intro:
 Be a little different, rather than a particular experiment and the results thereof, but rather observationsfrom multiple experiments and how they came together into an alternative method for knowing.

 Came to QS about 1.5 years ago with the objective of realising my potential. Though figuring out what I was and was not capableof.

 Standard approach
 Probably because of lack of any consideration, many of my initial experiments were carried out by using what I understood of the problem, background reading and rigorous tracking and this certainly works and is useful for a range of tasks. Though throughout several experiments where I experienced effectsthat I did not expect that were quite significant but not part of what I was originally measuring.
 Thinking about what is at the heart of the scientific method, it turns out to be rigour, not precisely what I was after, was trying to maximize results. Makes little sense to know you’re right if you’re in a worse position.


 Missed insight Payoff shortcomings rigour
 I don’t particularly want to go back to mental fog to show rigorously that yes it was my previous diet that was the heart of most of my issues. Was there a way that I could use these accidental insights in order to generate knowledge without falling prey to pitfalls of false positives?

 Need to guess ahead of time what you are looking for . I was looking for anything that moved the needle into the positive.
 Generally focused on rigour, keep as much constant as possible, to reduce noise. Long times and crossover groups.
 Still limited scientific use as n=1 and timing issues and just random noise.

 If looking at the wrong variable could lead to issues.
 Diet -> weight (OK)
 Also good mental performance
 Nootropics -> mental performance (problematic)
 Mental perf up, though also resulted in negative side-effectssuch as earlier sleepiness and jaw tension as well as a bit of a stutter I had not previously suffered from

 Potential solution just to measure everything. Takesup a lot of time, and effort, also likely to incorporate false positives just via sheer volume of variables being experimented on and random chance.

 Relevant XKCD

 On the subject of significance, you experience an outlier event, something that by definition lies outside normal experience. How significant is this? If you look at the definition and what has happened, significance is met by any stretch, because if nothing has changed, then nothing could have caused this… (or at least the chance that it occurs due to random chance is due to error when trying to calculate extremely small events.)

 This single event speaksvolumes, however.
 Turkey problem.
 One perfect day.

 Can’t capture this if you’re rating days on 1-5. As the richness of this datapoint gets lost if you just bin it with the top 20%.

 Outliers in research

 Assuming study was done correctly and has sufficient sample size. All rats in the control group died. Whereas a significant portion of the treatment group survived.

 Advantages.
 Track rather trivial, though still really bothering events.
 Track rare events impromptu.
 Richness of data to compare future cases to
 Faster, easier to track
 Easier to represent mentally, ie weak belief that X->Y rather than trying to remember statistical backing and small differences. Compare extreme novel experience with abstract reasoning.
 Ie this is unhealthy. Vs If I eat this I will be wokenup in the night with a really dry throat and have to fumble around in the dark to refill my bedside cup and will also have a really crappy day.
 Doesn’t require a credible explanation for why this has happened.

 Only valid in cases where experience is significantly different
 Prone to false positives, rather difficult to evaluate accuracy of any association generated.
 Still plagued by uncontrolled variables, ie tained cup. Rollback of changesdoes not work.


 Outliers with the emwave2.
 While I felt subjectively calmer after having the device, it wasn’t until I had the experience after a review of me having a bad day on the emwave where it was stuck in red though I didn’t feel any different on the day. And then realising the next day (which I was green) that everything I had worked on the previous day was completely wrong and was to be scrapped.

 Back rash.
 Last year developed a back rash, without any identifiable cause. Weeping sores, very uncomfortable. Several chairs around my corner desk. I had changed the one I had been using without realising. Noted that random things like wearing the suit jacket, or using balm improved the experience. But not until I changed chairs, only since it was irritating to my back after it had the sore that it went away. Did not really connect the dots that time, but I had the same type of experience after moving floors. I had already forgotten about the initial incident, though I know that this was not a new issue. Looking up the
troubleshooting steps for the previous incident informed my response here. Taking down my old chair solved the issue. Convinced it was not a fluke, since recurrence was unexpected.

 HEG neurofeedback.
 Training brain strengthening. Wanted just general increases to brain oxygenusage capacity. What started noticing, was that would remain in flow for much longer and not experience a clock-out at 4pm like previously and would often look at the clock to realise it was already past the time. Now realize that whenever I crash at 4 it is due to either stress such as lack of sleep or trying to juggle 2 input steams (podcast) with light work. Don’t know how often it would happen before, ‘grey area’ but aware it was a regular-ish issue. Now rare except in the above circumstances.

 Changing from Haverick to establishment 218 as much as I like the convienence and selection of the newer butcher. Found that I had experienced sinus issues. Congested nostrils sometime after switching. Don’t recall having them before and would have likely not realised as the products were roughly the same. GF beef, free range bacon. Tasted very similar. But noticeable deleteriouseffects. Still don’t know what in the baconprecisely that was causing the issues. Not really wanting to find out. Recall having the same reaction to beef broth. Guess it would be a shared ingredient. Reading labels. MSG comesto
mind, but unsure as there are likely more than 1 shared and it isn’t that unique.