Well, briefly put, it is where you found the information. You note it down so that someone else can go to the place you mentioned, see what you found, and match it with their own documentation. You note it down so that, later, you can see several seemingly-conflicting pieces of information and make a judgment call as to which is more likely to be real/true. For example:
Example A: John Smith is baptised on 1 January 1795, source - a book somebody wrote in 1995 about the SMITH name, mentioning John in passing
Example B: John Smith is baptised on 2 February 1790, source - church baptismal register of 1790
Now, which would be more confidence-inspiring to you? The book, written two hundred years after the fact, which only mentions him in passing? Or the official baptismal register, written at the time, probably by the man who actually got his fingers wet at the font and had to sharpen his quill in order to write? The very worst thing that could happen is that you look at your computer program, where it just says 'John SMITH, bap 3 June 1797' (with no source given) - and you wonder where on earth you found THAT information. (And, of course, you don't throw away the reference to the book. You put it in your notes, in case you want to refer to the book later. You just don't necessarily use it as the one-and-only source.)
And, believe me, it is so much easier to cite your sources right from the word 'go', rather than realising when you have been researching for ten years that you now have to remember what you were doing ten years ago and write several thousand sources...