Requirements Analysis

As we all know, today is the Qixi Festival.Those who can live the festival must have gone out to play and on the way to go out to play.In the afternoon, I also received a guide "How to shorten the distance between couples watching horror movies on Qixi Festival", unfortunately, although the guide is good, it seems that I can't use it

You can't go out and play without watching a movie, right?So, which movie is better to watch?Presumably many couples may not pay attention to the news of movies on weekdays. When they are looking for a movie to watch, they don't know what to watch.Maybe, I heard that the recent movie called "Lone on the Moon" has exceeded the box office of hundreds of millions. Should you watch this movie?

I don't know if you ask me.However, if you want to know which movies are better to watch, a good way is to check the ratings of the movies, so that you can see at a glance?Although a movie with a high rating does not mean that it is good, at least it means that most people like to watch it, right?Just do it, today we will crawl the ratings of the movie

Implementation Analysis

Crawler programs are like this. If there is anti-crawling, it will be more difficult.It would be easier if there was no back-climbing.In terms of data extraction, if the structure is complex and the required content is messy, it will be difficult to extract. If it is structured data, such as an ajax request, it will be very easy to get json data directly

It just so happens that today's movie ratings have neither anti-climbing nor data in json format, so they can be easily extracted at once, very easy.Of course, this is not our credit, mainly because the other party's website is more generous

As for the reading and writing of csv, it is also very easy, and there are many ways to achieve it.For example, you can read and write files by yourself, and read and write files in csv format.For example, read and write csv through a third-party library that can operate csv.Of course, there is a simpler way to directly use the built-in csv module in python.Although the function is not as powerful as the third-party library, it is also a good choice

Of course, there may be some friends who may not understand the csv format.In fact, csv is similar to excel, and it can also be made into this format of excel, but it is relatively simple. If you don't know this, you can learn about it separately

import csv # Import the built-in csv module# Through the context manager, open a file normallywith open("douban.csv", "w", newline="") as csvfile:# Through csv.writer(), create a writer for subsequent writing to the csv filew = csv.writer(csvfile, delimiter=",")# Through the method of writerow(), write the row of csvw.writerow(["title", "rating", "address", "picture"])for row in r.json()["subjects"]:w.writerow([row["title"], row["rate"], row["url"], row["cover"]])

In addition, some friends may also be confused.If you want to read and parse json files, shouldn't you import the json module?Why is this not used at all in the code below?

In fact, in requests, the json module is already included, and there is a simpler way to use it, so you can use json directly through requests.There is no need to introduce additional built-in modules for json

Complete code implementation

import requestsimport csvfrom base64 import b64decodeheaders = {"user-agent": "Mozilla/5.0"}url = b64decode("aHR0cHM6Ly9tb3ZpZS5kb3ViYW4uY29tL2ovc2VhcmNoX3N1YmplY3RzP3R5cGU9bW92aWUmdGFnPeeDremXqCZzb3J0PXJlY29tbWVuZCZwYWdlX2xpbWl0PTIwJnBhZ2Vcode).defr = requests.get(url, headers=headers)with open("douban.csv", "w", newline="") as csvfile:w = csv.writer(csvfile, delimiter=",")w.writerow(["title", "rating", "address", "picture"])for row in r.json()["subjects"]:w.writerow([row["title"], row["rate"], row["url"], row["cover"]])

After the program runs, you can see that it has been successfully saved as a csv file, the effect is as follows

So, finally, I wish everyone a happy Qixi Festival. If you can celebrate the festival, you can go to the movies according to the ratings.If you can't have a festival.Well, well, you can also go to the movies by yourself, otherwise, if you don't go out, won't everyone know that you have no festivals!