I’m scraping some websites and I’ve come across a site with a weird link structure. The <a href> shows a relative link, but that relative link added onto the current link does not equal the absolute link I see on the bottom of my browser.
[URL=http://g.imageshack.us/thpix.php]
If I want to make a robust web crawler, how do I detect that a certain site will have such an abnormality?
