Great idea.
Done a little digging, and if you get your TV through a TV card on your PC, then this might be what you want:
http://bemused.sourceforge.net/ - control your computer from your mobile with bluetooth
With a bit of clever programming you could probably get bemused to run lirc commands:
http://www.lirc.org - Linux Infrared Remote Control
or equivalent.
How you'd then wire this together is an issue of home interior design, but if you get it working, take videos. Live the dream.