Fluff mystery...

I have mystery fluff accumulating in my pencil. It's a light grey that I have no idea how it gets there. It's like pocket fluff or the fluff you get under optical mouse.

Will need to monitor how quickly it builds up.