ทำการ crawling data scrapy จากหน้าเว็บ ผ่าน os ubuntu

[golfreeze]

##Reference link
https://gist.github.com/vijayanandrp/e01cceb82a90ceaa54c671a70780bd25
https://doc.scrapy.org/en/latest/intro/tutorial.html

วันนี้มีโจทย์ให้ทำการลง scrapy เพื่อ crawling data จากหน้าเว็บ

# When the cache is clear, pip is working again.
hash -r

1. sudo apt-get install python3 python-dev python3-dev build-essential libssl-dev libffi-dev libxml2-dev libxslt-dev python3-pip

2. sudo pip3 install virtualenvwrapper

3. workon [try this command in terminal if it not works go to point 4]

4. source /usr/local/bin/virtualenvwrapper.sh (or)  source ~/.local/bin/virtualenvwrapper.sh   

5. workon [try this command in terminal, it will work definitely]

6. mkvirtualenv --python=python3 scrapy.py3 # Create a environment variable for scrapy project

7. pip3 install scrapy

8. scrapy

9. scrapy version -v

10. scrapy shell http://scrapy.org

11. deactivate

12. rmvirtualenv venv # To delete the virtual environment variable

scrapy shell http://scrapy.org