A Happy and lightweight Python Package that Provides an API to search for articles on Google News and returns a JSON response.

Hello,

Thanks for providing this piece of code.

I have recently come across weird behavior regarding the period parameter (e.g. 7d, you can get news from weeks prior). More importantly, the number of news output have dramatically reduced recently when combining countries and languages or even just providing a language and leaving the country parameter to None (for English)

Turning language parameter to any other language (e.g. French ['fr']) returns 0 articles systematically even for popular searches.

I suspect Google has changed/updated their url format and/or available countries/languages !

Been working on some code for past week and it had been working fine with get_news("topic") . Stopped working earlier get_top_news() still works. Tried using other keywords for topic but still returns nothing.

Any debug help?

This is my test code. It working with single keyword. Now i tried to make it with multiple keyword. It possible to do that? Example :

google_news = GNews(language='vi', country='Vietnam',
                    period='1h', max_results=20)
json_resp = google_news.get_news('Covid', 'Apple')
print(json_resp)

get_news('key') can search within a date range, if provided. Other functions return warnings if a date range has been provided as they do not support searching in this way. A workaround for each other function is suggested, but will provide slightly different results

Nice!

There doesn't currently appear to be a way to get news stories without specifying a topic (key). Could I modify the function so if the user uses get_news() without a key, or passing None or the empty string, it just grabs the top stories from the main feed for your locale?

Do you support Category or Location based searches?

Thanks!

⚔️ Things changed:

This PR primarily focuses on .github/workflows/python-publish.yml.

• The workflow now only triggers upon manual dispatch / a successful published release (which the comments at the beginning said that it did but actually didn't).

• The PyPI publish workflow doesn't require multiple dependencies / commands to be set up anymore. The build job uses the build Python package and the publishing workflow has been changed to the official one provided by the Python Packaging Authority.

• Bumped dependency version for checking out source code.

• Bumped dependency version for setting up Python.

🔖 To make this work:

Since the publishing workflow has been changed, you will need to remove these secrets from the repository:

• PYPI_USERNAME
• PYPI_PASSWORD

... and replace them with PYPI_API_TOKEN. This secret will contain a token provided by PyPI itself, which you can get from the Manage page of your project by clicking on "Create a token for project-name".

I hope this helps :D

The call to utils.import_or_install() introduced several issues:

• It was called with the parameter "newspaper3k". Since the correct import name is "newspaper" ("newspaper3k" is used for installation purposes), the __import __ call always failed, which means every call to the get_full_article() method would result in a redundant "pip install" process starting.

• Just to reiterate: Every user of this library right now sees a long and cryptic pip install output message WITH EACH AND EVERY CALL they make to get_full_article().

• Installing pip packages and/or modifying a user's environment without permission or any indication of such behavior (nothing in the docs) is unacceptable.

• Installing packages using direct calls to pip module internals is NOT the way to install packages and also yields warnings regarding the usage of incorrect pip wrappers.

Hello. I have some doubts about this line https://github.com/ranahaani/GNews/blob/master/gnews/gnews.py#L86. I need to run GNews in a docker container by using Airflow in order to get information about articles. I got the following message:

WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.

And then Airflow sent me a negative signal to fail my task.

hi, it seems that I can't get more than 100 results despite changing the max_Result, why is that and would I get different results if I repeat the search ?

Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

Changes included in this PR

• Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
  • requirements.txt

requests 2.26.0 requires certifi, which is not installed.

Vulnerabilities that will be fixed

By pinning:

Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- | 626/1000
Why? Recently disclosed, Has a fix available, CVSS 6.8 | Insufficient Verification of Data Authenticity
SNYK-PYTHON-CERTIFI-3164749 | certifi:
2021.10.8 -> 2022.12.7
| No | No Known Exploit

(*) Note that the real score may have changed since the PR was raised.

Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

Check the changes in this PR to ensure they won't cause issues with your project.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

For more information: 🧐 View latest project report

🛠 Adjust project settings

📚 Read more about Snyk's upgrade and patch logic

Learn how to fix vulnerabilities with free interactive lessons:

🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

This PR was automatically created by Snyk using the credentials of a real user.

Changes included in this PR

• Dockerfile

We recommend upgrading to python:3.12.0a3, as this image has only 272 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.

Some of the most important vulnerabilities in your base image include:

| Severity | Priority Score / 1000 | Issue | Exploit Maturity | | :------: | :-------------------- | :---- | :--------------- | | | 714 | Directory Traversal
SNYK-DEBIAN11-DPKG-2847942 | No Known Exploit | | | 714 | Out-of-bounds Read
SNYK-DEBIAN11-LIBTASN16-3061097 | No Known Exploit | | | 714 | OS Command Injection
SNYK-DEBIAN11-OPENSSL-2807596 | No Known Exploit | | | 714 | OS Command Injection
SNYK-DEBIAN11-OPENSSL-2933518 | No Known Exploit | | | 614 | Improper Input Validation
SNYK-DEBIAN11-XZUTILS-2444276 | No Known Exploit |

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

For more information: 🧐 View latest project report

🛠 Adjust project settings

Learn how to fix vulnerabilities with free interactive lessons:

🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

I am trying to obtain news reports within a specific date range, using start_date and end_date parameters, but the ranging doesn't seem to work. It fetches the top news reports from the current date only. I have also attached the code and results image below. I have tried both the tuple approach as well as the datetime object approach, but none seem to work. I have also pointed to particular piece of code, which could have not set the parameter for end date.

For some reason, I cannot seem to fetch any news anymore, not even with the README example. Could it be an IP issue? It seems to be working when using VPN.

I using GNews get_full_article() function to extract the top_image from the Article. However, when I run this on my production server it throws me the below error:

ERROR: Articledownload()failed with HTTPSConnectionPool(host='indianexpress.com', port=443): Max retries exceeded with url: /article/idea-exchange/gautam-gambhir-idea-exchange-first-challenge-mcd-polls-change-narrative-bjp-doesnt-do-anything-8158944/ (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 403 Forbidden'))) on URL https://indianexpress.com/article/idea-exchange/gautam-gambhir-idea-exchange-first-challenge-mcd-polls-change-narrative-bjp-doesnt-do-anything-8158944/

I searched through Google and ended up with this solution:

from newspaper import Article
from newspaper import Config

user_agent = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Firefox/78.0'
config = Config()

config.browser_user_agent = user_agent

url = "https://www.chicagotribune.com/nation-world/ct-florida-school-shooter-nikolas-cruz-20180217-story.html"

page = Article(url, config=config)


page.download()
page.parse()
print(page.text)

As per the code above, I need to mention the user agentand get it assigned to config.browser_user_agent to prevent the server from getting banned. However, if I want to use gnews.get_full_article() I am not able to specify the config parameter inside. Is there any provision to mention this parameter? Am I missing something?

Hi,

I use GNews a lot and lately I've been having some trouble with it.. The code I use is: google_news.get_news("News I want to get"). Normally I get several hundred links a day, now only a few? It seems that he can't find anything anymore, because even if I use the code once for a specific search term, it doesn't find all the news by far?

I've been using the same code for a long time and nothing has changed in the code. I also did not change the version of GNews. I suddenly just got a lot less links?

First off, thanks for creating and releasing this very helpful Package, it saved me a lot of time from coding it on my own for my quick project.

The only suggestion I have is in reference to the formatting of the return value for 'description' What gets returned to me is not a description of the article but a series of short titles from other news sites, without links. I know you are running it through BeautifulSoup which removes the links and the list structure and what is left is a confusing mess.

I modified the code so that I get back everything I want, but for other users you may want to add an option to switch that on and off. I added this to my base and control it during initialization of GNews, now I switch formatting by BeautifulSoup on/off with a simple option I pass once anytime it's needed. Considering most consumers of your API are technical this will not be confusing.