It could be an optical illusion caused by some unknown factor...
or it could very well be a ghost.
I am curious if you've encountered anything else that would give you the impression of a haunting.
I
would recommend taking down a full description of what this apparition
looked like. From there I would research not only the history of the
house, but of the property and the surrounding area. It could give you
clues as to what you are dealing with. You may even be able to identify
the person.
If nothing in your house seems malevolent, its probably nothing to be too concerned about. You mentioned that you thought it was haunted for years. If nothing terrible has happened then you're probably good. Ghosts range from malevolent to benevolent with a vast mid-section of aloof voyeurs. They're like analog internet stalkers who know where you live but can't do anything about it.
If
you are concerned you can always contact a medium. Just make sure the
person is well-respected. Some of them have a true gift. Others are
totally nuts.
You can also have the house blessed by a member of the clergy.
I hope that can be of use.