twisted python failure class openssl images not downloading

Тема в разделе "Perl, Python, Ruby", создана пользователем matty2rf, 14 сен 2020.

Метки:
  1. matty2rf

    matty2rf Создатель

    Регистр.:
    12 окт 2018
    Сообщения:
    26
    Симпатии:
    2
    I have a great scraper which working well for the last 3 years however all of sudden i found it is no longer downloading the images to my local hard drive - just the image url is there in the csv file it generates at the end of the scrape same is with the pdf file,

    I checked online and saw some posts and tried doing this did not work still same error showing when scraper runs

    context.py:

    from OpenSSL import SSL
    from scrapy.core.downloader.contextfactory import ScrapyClientContextFactory


    class CustomContextFactory(ScrapyClientContextFactory:(
    """
    Custom context factory that allows SSL negotiation.
    """

    def __init__(self:(
    # Use SSLv23_METHOD so we can use protocol negotiation
    self.method = SSL.SSLv23_METHOD
    settings.py

    DOWNLOADER_CLIENTCONTEXTFACTORY = 'yourproject.context.CustomContextFactory'


    Another thing i tried was update twisted but then i got other errors and the scraper does not work at all
    Twisted 20.3.0

    my twisted version is 8. and the scraper only seems to work with this version
     
    Последнее редактирование модератором: 17 сен 2020