Golfreeze.packetlove.com: Life style of Golfreeze Canon400D Family kammtan.com Jazz Freebsd Unix Linux System Admin guitar Music
เกี่ยวกับ Cloud computing + Big data ทั้งหมด => เกี่ยวกับ Cloud Computing Openstack , AWS , Rackspace knowledge , DigitalOcean , Ar-bro.com => Topic started by: golfreeze on มีนาคม 07, 2018, 01:52:38 PM
-
##Reference link
https://gist.github.com/vijayanandrp/e01cceb82a90ceaa54c671a70780bd25
https://doc.scrapy.org/en/latest/intro/tutorial.html
วันนี้มีโจทย์ให้ทำการลง scrapy เพื่อ crawling data จากหน้าเว็บ
# When the cache is clear, pip is working again.
hash -r
1. sudo apt-get install python3 python-dev python3-dev build-essential libssl-dev libffi-dev libxml2-dev libxslt-dev python3-pip
2. sudo pip3 install virtualenvwrapper
3. workon [try this command in terminal if it not works go to point 4]
4. source /usr/local/bin/virtualenvwrapper.sh (or) source ~/.local/bin/virtualenvwrapper.sh
5. workon [try this command in terminal, it will work definitely]
6. mkvirtualenv --python=python3 scrapy.py3 # Create a environment variable for scrapy project
7. pip3 install scrapy
8. scrapy
9. scrapy version -v
10. scrapy shell http://scrapy.org
11. deactivate
12. rmvirtualenv venv # To delete the virtual environment variable
scrapy shell http://scrapy.org