The crawler returns null when parsing with XPath. The reason why the crawler cannot get the corresponding element and the solution

Parsing web pages while writing crawlers , The most used parsing method is xpath analysis , But in use, in use xpath When parsing , I wrote it myself xpath The statement is correct , But the return value is still empty

The reason is usually some anti climbing measures taken by the front end , When writing a web page, you usually omit a layer of tags , But the omitted tags will be automatically added by the browser , Change to the correct structure ..
When we check through the browser , The code structure you see has been modified by the browser , What the crawler gets is the source code
So according to the revised xpath Parsing the source code will not find the corresponding element , The return value is naturally empty

give an example
Browser modified code

xpath sentence '/html/body/div[5]/div[3]/div[2]/table/tbody/tr[1]/td[2]/a/@href'

Source code
Lack of one tbody label ,
xpath/html/body/div[5]/div[3]/div[2]/table/tr[1]/td[2]/a/@href take taody Delete

summary When using xpath When the corresponding element cannot be obtained , Look at the source code structure , Analyze according to the source code