Centos7 Python3下安装scrapy的详细步骤_Python

苦逼的前夜

昨晚很辛苦，搞到晚上快两点，最后还是没有把python3下的scrapy框架安装起来，后面还把yum这玩意给弄坏了，一直找不到命令。今天早上又自己弄了快一上午，又求助@函兮，弄了快一个中午，最后无奈还是没有弄好yum跟python这玩意，最后还是放弃治疗了。真的是什么招，什么损招都用完了，最后也没折了，直接报告老大去，然后把阿里云的centos7实例重新磁盘回滚了一下。

正确的安装姿势

这个不多废话，如果你是直接接手过来一个centos7的实例镜像，当然包括腾讯云或者阿里云上面的，只要按照下面的步骤进行安装，没有装不上，只有装得舒服，不到半小时，完完整整的装好Python3，装好Python3下的scrapy框架

步骤

前言：不用改掉系统python2.7 ，原来是python2.7，我们还进行python2.7的保留

1.编译前准备其他库的安装(使用sudo命令)

(1)

1	`[root@hadron ~]# yum -y groupinstall "Development tools"`

(2)

复制代码代码如下:

	

	[root@hadron ~]# yum -y install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel

(2)这里有点长，一定要复制完整了再进行回车，中途可能会报错，提示会有：--skip-broken

可以使用命令：

复制代码代码如下:

	

	[root@hadron ~]# yum -y install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel --skip-broken

2 . 到这里我们就可以进行python3.5的源码下载了

(1)使用wget命令：

1	`[root@hadron ~]# wget https://www.python.org/ftp/python/3.5.5/Python-3.5.5.tgz`

（2）然后解压缩，进入根目录

1 2	`[root@hadron ~]# tar -zxvf Python-3.5.5.tgz` `[root@hadron ~]# cd Python-3.5.5/`

（3）python3.5进行编译安装

1 2	`[root@hadron ~]# ./configure --prefix=/usr/local/python3` `[root@hadron ~]# make && make install`

（4）创建Python3，pip3的软链接

1 2	`[root@hadron ~]# ln -s /usr/local/python3/bin/python3 /usr/bin/python3` `[root@hadron ~]# ln -s /usr/local/python3/bin/pip3 /usr/bin/pip3`

（5）验证Python3 和pip3 是否安装成功

									[root@hadron ~]# python

									Python 2.7.5 (default, Nov 6 2016, 00:28:07) 

									[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2

									Type "help", "copyright", "credits" or "license" for more information.

									>>> quit()

									[root@hadron ~]# python3

									Python 3.5.5 (default, Feb 27 2018, 09:28:49) 

									[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux

									Type "help", "copyright", "credits" or "license" for more information.

									>>> quit()

									[root@hadron ~]# 

									[root@hadron ~]# pip3 --version

									pip 9.0.1 from /usr/local/python3/lib/python3.5/site-packages (python 3.5)

这里我们可以看到默认的python还是系统自带的python2.7，而python3就是我们解压的python3.5.5了，pip3 也是python3.5下的pip了。

上面我们已经把python3跟pip3安装好了，接下来，我们进行Python3下的Scrapy框架安装

3 安装Scrapy爬虫框架

（1）安装Scrapy

									[root@hadron ~]# pip3 install scrapy

									Collecting scrapy

									 Downloading Scrapy-1.5.0-py2.py3-none-any.whl (251kB)

									  100% |████████████████████████████████| 256kB 1.1MB/s

									Requirement already satisfied: lxml in /usr/local/python3/lib/python3.5/site-packages (from scrapy)

									Collecting PyDispatcher>=2.0.5 (from scrapy)

									 Downloading PyDispatcher-2.0.5.tar.gz

									....

									....

									Collecting pycparser (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=2.1.4->pyOpenSSL->scrapy)

									 Downloading pycparser-2.18.tar.gz (245kB)

									  100% |████████████████████████████████| 256kB 339kB/s

									Installing collected packages: PyDispatcher, zope.interface, constantly, incremental, six, attrs, Automat, hyperlink, Twisted, cssselect, w3lib, parsel, asn1crypto, pycparser, cffi, cryptography, pyOpenSSL, pyasn1, pyasn1-modules, service-identity, queuelib, scrapy

									 Running setup.py install for PyDispatcher ... done

									 Running setup.py install for Twisted ... done

									 Running setup.py install for pycparser ... done

									Successfully installed Automat-0.6.0 PyDispatcher-2.0.5 Twisted-17.9.0 asn1crypto-0.24.0 attrs-17.4.0 cffi-1.11.4 constantly-15.1.0 cryptography-2.1.4 cssselect-1.0.3 hyperlink-18.0.0 incremental-17.5.0 parsel-1.4.0 pyOpenSSL-17.5.0 pyasn1-0.4.2 pyasn1-modules-0.2.1 pycparser-2.18 queuelib-1.4.2 scrapy-1.5.0 service-identity-17.0.0 six-1.11.0 w3lib-1.19.0 zope.interface-4.4.3

									[root@hadron ~]#