github (https://github.com/tensorflow/models/tree/master/tutorials/image/cifar10) 에서 cifar10.py, cifar10_input.py, cifar10_train 를 다운 받은 후 cifar10_train.py 를 수행
2018년 1월 22일 월요일
2017년 12월 18일 월요일
docker 기본 사용 - Ubuntu
설치된 docker engine 확인
u0018670@sys-90572:~$ dpkg -l |grep docker
ii docker.io 1.13.1-0ubuntu1~16.04.2 ppc64el Linux container runtime
ii runc 1.0.0~rc2+docker1.13.1-0ubuntu1~16.04.1 ppc64el Open Container Project - runtime
아무런 docker image 가 없는 것을 확인
u0018670@sys-90572:~$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
그래서, ppc64le/ubuntu docker image 를 실행시키면 latest 로 docker hub 로 부터 docker tag 가 latest 인 것을 download 후 docker 실행
u0018670@sys-90572:~$ sudo docker run -it ppc64le/ubuntu
Unable to find image 'ppc64le/ubuntu:latest' locally
latest: Pulling from ppc64le/ubuntu
586a07d977d9: Pull complete
2bf911559ec0: Pull complete
48999c060756: Pull complete
fa5e5494889f: Pull complete
3bd4f80d03d3: Pull complete
Digest: sha256:79d11934f87adb78614d5ca93092f6681ca154c2bd9df56a7027615c56d53e41
Status: Downloaded newer image for ppc64le/ubuntu:latest
root@b916c38bbbb7:/# ls <== docker container 안
bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var
root@b916c38bbbb7:/# exit
exit
생성된 docker image 확인
u0018670@sys-90572:~$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ppc64le/ubuntu latest 44c2eaa37868 3 days ago 150 MB
수행중인 container 확인
u0018670@sys-90572:~$ sudo docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b916c38bbbb7 ppc64le/ubuntu "/bin/bash" 54 seconds ago Exited (0) 17 seconds ago epic_cori
exit 한 container 재시작
u0018670@sys-90727:~$ sudo docker start confident_euclid
confident_euclid
u0018670@sys-90727:~$ sudo docker attach confident_euclid
root@0f647ddac01a:/#
수행중인 container 를 image 로 commit
u0018670@sys-90727:~$ sudo docker commit 58d0d814ae31 ppc64le/ubuntu:latest
sha256:4de2af599de3a7a0aa2351a9fcb6e9b8cbaa8f3a345d6fed03291477097da82d
u0018670@sys-90727:~$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ppc64le/ubuntu latest 4de2af599de3 About a minute ago 260MB
ppc64le/ubuntu 16.04 44c2eaa37868 3 weeks ago 150MB
수행중인 container 삭제
u0018670@sys-90572:~$ sudo docker rm epic_cori
epic_cori
u0018670@sys-90572:~$ sudo docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
u0018670@sys-90572:~$
한꺼번에 모든 container 삭제는 docker rm $(docker ps -a -q)
다음은 2개의 container 가 시작된 상황. (ppc64le/ubuntu 이미지로 eager_hugle, quirky_pike)
u0018670@sys-90572:~$ sudo docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
eaf515d4a2c6 ppc64le/ubuntu "/bin/bash" 16 seconds ago Exited (0) 12 seconds ago eager_hugle
4e3e7338019b ppc64le/ubuntu "/bin/bash" About a minute ago Exited (0) 21 seconds ago quirky_pike
container 수행 중이기에 삭제 되지 않는 docker image, rm 옵션으로 container 를 먼저 삭제 후 image 삭제
u0018670@sys-90572:~$ sudo docker rmi ppc64le/ubuntu
Error response from daemon: conflict: unable to remove repository reference "ppc64le/ubuntu" (must force) - container 4e3e7338019b is using its referenced image 44c2eaa37868
u0018670@sys-90572:~$
docker hub 에 생성된 이미지 upload
u0018670@sys-90461:~/data$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
leearmy/cudnn6 ppc64le de632816acda About an hour ago 6.58GB
leearmy/ubuntu 16.04 23296ffbcbfc 4 hours ago 359MB
ppc64le/ubuntu 16.04 44c2eaa37868 3 days ago 150MB
다음 명령어를 수행하기 위해서는 먼저 https://hub.docker.com/ 에 계정과 repository 를 생성하여야 함
u0018670@sys-90461:~/data$ sudo docker login --username=leearmy
Password:
Login Succeeded
u0018670@sys-90461:~/data$ sudo docker push leearmy/cudnn6:ppc64le
The push refers to a repository [docker.io/leearmy/cudnn6]
026ec1f6d915: Pushed
c963a10dda0d: Pushed
4a08cdc31ed0: Pushed
49076433d00c: Pushed
8788fee75149: Pushed
d9561c8b4bf3: Pushed
54ac221998f0: Pushed
b10475677e3b: Pushed
adf3a2526761: Pushed
e8449de7ace1: Pushed
0c5b7df1df7c: Pushed
b1dc141f6073: Mounted from ppc64le/ubuntu
448bb1e132fc: Mounted from ppc64le/ubuntu
e7f4651b678d: Mounted from ppc64le/ubuntu
ee8f1c150908: Mounted from ppc64le/ubuntu
258cc4d5502c: Mounted from ppc64le/ubuntu
ppc64le: digest: sha256:3df7ee55f48c08fe7dc836c60053867dac28ada8b7eb9b9413b09e5361e4937c size: 3690
다음과 같이 https://hub.docker.com/ 에서 올려진 이미지 확인
docker hub 에 올려진 이미지는 다른 서버에서 pull
u0018670@sys-90572:~$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ppc64le/ubuntu latest 44c2eaa37868 3 days ago 150 MB
u0018670@sys-90572:~$ sudo docker pull leearmy/cudnn6:ppc64le
ppc64le: Pulling from leearmy/cudnn6
586a07d977d9: Already exists
2bf911559ec0: Already exists
48999c060756: Already exists
fa5e5494889f: Already exists
3bd4f80d03d3: Already exists
3d482c0c4d09: Pull complete
9e891e52327a: Pull complete
a40daf5d9d7f: Pull complete
2ca681ce2118: Pull complete
111379f9235b: Pull complete
9db8b0fb9301: Pull complete
b2b39482e9b8: Pull complete
93f5e3ec7d59: Pull complete
38e0ece0a1d4: Pull complete
dbc7427ab4ea: Pull complete
f1cc2b939ac9: Pull complete
Digest: sha256:3df7ee55f48c08fe7dc836c60053867dac28ada8b7eb9b9413b09e5361e4937c
Status: Downloaded newer image for leearmy/cudnn6:ppc64le
u0018670@sys-90572:~$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
leearmy/cudnn6 ppc64le de632816acda 20 hours ago 6.58 GB
ppc64le/ubuntu latest 44c2eaa37868 3 days ago 150 MBu0018670@sys-90572:~$
u0018670@sys-90572:~$ dpkg -l |grep docker
ii docker.io 1.13.1-0ubuntu1~16.04.2 ppc64el Linux container runtime
ii runc 1.0.0~rc2+docker1.13.1-0ubuntu1~16.04.1 ppc64el Open Container Project - runtime
아무런 docker image 가 없는 것을 확인
u0018670@sys-90572:~$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
그래서, ppc64le/ubuntu docker image 를 실행시키면 latest 로 docker hub 로 부터 docker tag 가 latest 인 것을 download 후 docker 실행
u0018670@sys-90572:~$ sudo docker run -it ppc64le/ubuntu
Unable to find image 'ppc64le/ubuntu:latest' locally
latest: Pulling from ppc64le/ubuntu
586a07d977d9: Pull complete
2bf911559ec0: Pull complete
48999c060756: Pull complete
fa5e5494889f: Pull complete
3bd4f80d03d3: Pull complete
Digest: sha256:79d11934f87adb78614d5ca93092f6681ca154c2bd9df56a7027615c56d53e41
Status: Downloaded newer image for ppc64le/ubuntu:latest
root@b916c38bbbb7:/# ls <== docker container 안
bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var
root@b916c38bbbb7:/# exit
exit
생성된 docker image 확인
u0018670@sys-90572:~$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ppc64le/ubuntu latest 44c2eaa37868 3 days ago 150 MB
수행중인 container 확인
u0018670@sys-90572:~$ sudo docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b916c38bbbb7 ppc64le/ubuntu "/bin/bash" 54 seconds ago Exited (0) 17 seconds ago epic_cori
exit 한 container 재시작
u0018670@sys-90727:~$ sudo docker start confident_euclid
confident_euclid
u0018670@sys-90727:~$ sudo docker attach confident_euclid
root@0f647ddac01a:/#
수행중인 container 를 image 로 commit
u0018670@sys-90727:~$ sudo docker commit 58d0d814ae31 ppc64le/ubuntu:latest
sha256:4de2af599de3a7a0aa2351a9fcb6e9b8cbaa8f3a345d6fed03291477097da82d
u0018670@sys-90727:~$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ppc64le/ubuntu latest 4de2af599de3 About a minute ago 260MB
ppc64le/ubuntu 16.04 44c2eaa37868 3 weeks ago 150MB
수행중인 container 삭제
u0018670@sys-90572:~$ sudo docker rm epic_cori
epic_cori
u0018670@sys-90572:~$ sudo docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
u0018670@sys-90572:~$
한꺼번에 모든 container 삭제는 docker rm $(docker ps -a -q)
다음은 2개의 container 가 시작된 상황. (ppc64le/ubuntu 이미지로 eager_hugle, quirky_pike)
u0018670@sys-90572:~$ sudo docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
eaf515d4a2c6 ppc64le/ubuntu "/bin/bash" 16 seconds ago Exited (0) 12 seconds ago eager_hugle
4e3e7338019b ppc64le/ubuntu "/bin/bash" About a minute ago Exited (0) 21 seconds ago quirky_pike
container 수행 중이기에 삭제 되지 않는 docker image, rm 옵션으로 container 를 먼저 삭제 후 image 삭제
u0018670@sys-90572:~$ sudo docker rmi ppc64le/ubuntu
Error response from daemon: conflict: unable to remove repository reference "ppc64le/ubuntu" (must force) - container 4e3e7338019b is using its referenced image 44c2eaa37868
u0018670@sys-90572:~$
docker hub 에 생성된 이미지 upload
u0018670@sys-90461:~/data$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
leearmy/cudnn6 ppc64le de632816acda About an hour ago 6.58GB
leearmy/ubuntu 16.04 23296ffbcbfc 4 hours ago 359MB
ppc64le/ubuntu 16.04 44c2eaa37868 3 days ago 150MB
다음 명령어를 수행하기 위해서는 먼저 https://hub.docker.com/ 에 계정과 repository 를 생성하여야 함
u0018670@sys-90461:~/data$ sudo docker login --username=leearmy
Password:
Login Succeeded
u0018670@sys-90461:~/data$ sudo docker push leearmy/cudnn6:ppc64le
The push refers to a repository [docker.io/leearmy/cudnn6]
026ec1f6d915: Pushed
c963a10dda0d: Pushed
4a08cdc31ed0: Pushed
49076433d00c: Pushed
8788fee75149: Pushed
d9561c8b4bf3: Pushed
54ac221998f0: Pushed
b10475677e3b: Pushed
adf3a2526761: Pushed
e8449de7ace1: Pushed
0c5b7df1df7c: Pushed
b1dc141f6073: Mounted from ppc64le/ubuntu
448bb1e132fc: Mounted from ppc64le/ubuntu
e7f4651b678d: Mounted from ppc64le/ubuntu
ee8f1c150908: Mounted from ppc64le/ubuntu
258cc4d5502c: Mounted from ppc64le/ubuntu
ppc64le: digest: sha256:3df7ee55f48c08fe7dc836c60053867dac28ada8b7eb9b9413b09e5361e4937c size: 3690
다음과 같이 https://hub.docker.com/ 에서 올려진 이미지 확인
docker hub 에 올려진 이미지는 다른 서버에서 pull
u0018670@sys-90572:~$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ppc64le/ubuntu latest 44c2eaa37868 3 days ago 150 MB
u0018670@sys-90572:~$ sudo docker pull leearmy/cudnn6:ppc64le
ppc64le: Pulling from leearmy/cudnn6
586a07d977d9: Already exists
2bf911559ec0: Already exists
48999c060756: Already exists
fa5e5494889f: Already exists
3bd4f80d03d3: Already exists
3d482c0c4d09: Pull complete
9e891e52327a: Pull complete
a40daf5d9d7f: Pull complete
2ca681ce2118: Pull complete
111379f9235b: Pull complete
9db8b0fb9301: Pull complete
b2b39482e9b8: Pull complete
93f5e3ec7d59: Pull complete
38e0ece0a1d4: Pull complete
dbc7427ab4ea: Pull complete
f1cc2b939ac9: Pull complete
Digest: sha256:3df7ee55f48c08fe7dc836c60053867dac28ada8b7eb9b9413b09e5361e4937c
Status: Downloaded newer image for leearmy/cudnn6:ppc64le
u0018670@sys-90572:~$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
leearmy/cudnn6 ppc64le de632816acda 20 hours ago 6.58 GB
ppc64le/ubuntu latest 44c2eaa37868 3 days ago 150 MBu0018670@sys-90572:~$
2017년 12월 10일 일요일
IBM Minsky (S822LC for HPC) 에서 nvidia-docker 설치 - Ubuntu
ppc64le 환경에서 nvidia-docker 를 설치할때마다 nvidia-docker 가 update 되고 방법이 틀려졌음.
http://hwengineer.blogspot.kr/2017/02/minsky-ubuntu-1604-xenial-ppc64le.html
https://hwengineer.blogspot.kr/2017/10/ppc64le-nvidia-docker-201710.html
2017년 12월 10일 현재 nvidia-docker 를 설치하기 위해서는 다음 사이트를 참고하였음.
https://developer.ibm.com/linuxonpower/docker-on-power/
https://nvidia.github.io/nvidia-docker/
nvidia-docker 가 설치될 환경은
-.ubuntu 16.04
-.CUDA 8.0 + cudnn 6.0.21
먼저 docker engine 설치
test_svr@ubuntu:~$
test_svr@ubuntu:~$ sudo vi /etc/apt/sources.list.d/xenial-docker.list
deb http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el/ xenial main
test_svr@ubuntu:~$ sudo apt-get update
test_svr@ubuntu:~$ sudo apt-get install docker-ce
test_svr@ubuntu:~$ dpkg -l |grep -i docker
ii docker 1.5-1 ppc64el System tray for KDE3/GNOME2 docklet applications
ii docker-ce 17.06.0~ce-0~ubuntu ppc64el Docker: the open-source application container engine
docker engine 이 설치되었으면 nvidia-docker 설치
test_svr@ubuntu:~$ curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
> sudo apt-key add -
OK
test_svr@ubuntu:~$ curl -s -L https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el/nvidia-docker.list | \
> sudo tee /etc/apt/sources.list.d/nvidia-docker.list
deb https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el /
test_svr@ubuntu:~$ sudo apt-get update
Get:1 file:/var/cuda-repo-8-0-local-ga2v2 InRelease
Ign:1 file:/var/cuda-repo-8-0-local-ga2v2 InRelease
Get:2 file:/var/nvidia-driver-local-repo-tr6 InRelease
Ign:2 file:/var/nvidia-driver-local-repo-tr6 InRelease
Get:3 file:/var/cuda-repo-8-0-local-ga2v2 Release [574 B]
Get:4 file:/var/nvidia-driver-local-repo-tr6 Release [574 B]
Get:3 file:/var/cuda-repo-8-0-local-ga2v2 Release [574 B]
Get:4 file:/var/nvidia-driver-local-repo-tr6 Release [574 B]
Get:7 http://ports.ubuntu.com/ubuntu-ports xenial-security InRelease [102 kB]
Hit:8 http://kr.ports.ubuntu.com/ubuntu-ports xenial InRelease
Get:9 https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el InRelease [1,098 B]
Get:10 http://kr.ports.ubuntu.com/ubuntu-ports xenial-updates InRelease [102 kB]
Get:11 https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el Packages [704 B]
Ign:12 http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial InRelease
Get:13 http://kr.ports.ubuntu.com/ubuntu-ports xenial-backports InRelease [102 kB]
Hit:14 http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial Release
Ign:15 http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial Release.gpg
Fetched 308 kB in 2s (111 kB/s)
Reading package lists... Done
W: The repository 'http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial Release' is not signed.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
test_svr@ubuntu:~$ sudo apt-get install -y nvidia-docker
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
nvidia-docker
0 upgraded, 1 newly installed, 0 to remove and 84 not upgraded.
Need to get 2,267 kB of archives.
After this operation, 16.0 MB of additional disk space will be used.
Get:1 https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el nvidia-docker 1.0.1-1 [2,267 kB]
Fetched 2,267 kB in 2s (964 kB/s)
Selecting previously unselected package nvidia-docker.
(Reading database ... 122052 files and directories currently installed.)
Preparing to unpack .../nvidia-docker_1.0.1-1_ppc64el.deb ...
Unpacking nvidia-docker (1.0.1-1) ...
Processing triggers for ureadahead (0.100.0-19) ...
ureadahead will be reprofiled on next reboot
Setting up nvidia-docker (1.0.1-1) ...
Configuring user
Setting up permissions
Processing triggers for ureadahead (0.100.0-19) ...
test_svr@ubuntu:~$
http://hwengineer.blogspot.kr/2017/02/minsky-ubuntu-1604-xenial-ppc64le.html
https://hwengineer.blogspot.kr/2017/10/ppc64le-nvidia-docker-201710.html
2017년 12월 10일 현재 nvidia-docker 를 설치하기 위해서는 다음 사이트를 참고하였음.
https://developer.ibm.com/linuxonpower/docker-on-power/
https://nvidia.github.io/nvidia-docker/
nvidia-docker 가 설치될 환경은
-.ubuntu 16.04
-.CUDA 8.0 + cudnn 6.0.21
먼저 docker engine 설치
test_svr@ubuntu:~$
test_svr@ubuntu:~$ sudo vi /etc/apt/sources.list.d/xenial-docker.list
deb http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el/ xenial main
test_svr@ubuntu:~$ sudo apt-get update
test_svr@ubuntu:~$ sudo apt-get install docker-ce
test_svr@ubuntu:~$ dpkg -l |grep -i docker
ii docker 1.5-1 ppc64el System tray for KDE3/GNOME2 docklet applications
ii docker-ce 17.06.0~ce-0~ubuntu ppc64el Docker: the open-source application container engine
docker engine 이 설치되었으면 nvidia-docker 설치
test_svr@ubuntu:~$ curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
> sudo apt-key add -
OK
test_svr@ubuntu:~$ curl -s -L https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el/nvidia-docker.list | \
> sudo tee /etc/apt/sources.list.d/nvidia-docker.list
deb https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el /
test_svr@ubuntu:~$ sudo apt-get update
Get:1 file:/var/cuda-repo-8-0-local-ga2v2 InRelease
Ign:1 file:/var/cuda-repo-8-0-local-ga2v2 InRelease
Get:2 file:/var/nvidia-driver-local-repo-tr6 InRelease
Ign:2 file:/var/nvidia-driver-local-repo-tr6 InRelease
Get:3 file:/var/cuda-repo-8-0-local-ga2v2 Release [574 B]
Get:4 file:/var/nvidia-driver-local-repo-tr6 Release [574 B]
Get:3 file:/var/cuda-repo-8-0-local-ga2v2 Release [574 B]
Get:4 file:/var/nvidia-driver-local-repo-tr6 Release [574 B]
Get:7 http://ports.ubuntu.com/ubuntu-ports xenial-security InRelease [102 kB]
Hit:8 http://kr.ports.ubuntu.com/ubuntu-ports xenial InRelease
Get:9 https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el InRelease [1,098 B]
Get:10 http://kr.ports.ubuntu.com/ubuntu-ports xenial-updates InRelease [102 kB]
Get:11 https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el Packages [704 B]
Ign:12 http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial InRelease
Get:13 http://kr.ports.ubuntu.com/ubuntu-ports xenial-backports InRelease [102 kB]
Hit:14 http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial Release
Ign:15 http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial Release.gpg
Fetched 308 kB in 2s (111 kB/s)
Reading package lists... Done
W: The repository 'http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial Release' is not signed.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
test_svr@ubuntu:~$ sudo apt-get install -y nvidia-docker
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
nvidia-docker
0 upgraded, 1 newly installed, 0 to remove and 84 not upgraded.
Need to get 2,267 kB of archives.
After this operation, 16.0 MB of additional disk space will be used.
Get:1 https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el nvidia-docker 1.0.1-1 [2,267 kB]
Fetched 2,267 kB in 2s (964 kB/s)
Selecting previously unselected package nvidia-docker.
(Reading database ... 122052 files and directories currently installed.)
Preparing to unpack .../nvidia-docker_1.0.1-1_ppc64el.deb ...
Unpacking nvidia-docker (1.0.1-1) ...
Processing triggers for ureadahead (0.100.0-19) ...
ureadahead will be reprofiled on next reboot
Setting up nvidia-docker (1.0.1-1) ...
Configuring user
Setting up permissions
Processing triggers for ureadahead (0.100.0-19) ...
test_svr@ubuntu:~$
2017년 12월 7일 목요일
내가 자주 쓰는, 기억해야 하는 파이썬 command - Ubuntu
framework 버전 확인
anaconda 를 통해 pip install 로 설치된 tensorflow 1.3.1 로 동작 확인
u0018670@sys-90257:~$ env | grep PATH
PATH=/home/u0018670/bin:/home/u0018670/.local/bin:/home/u0018670/anaconda2/bin:/opt/ibm/xlC/current/bin:/opt/ibm/xlf/current/bin:/opt/at10.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
u0018670@sys-90257:~$ python
Python 2.7.13 |Anaconda, Inc.| (default, Sep 15 2017, 20:54:27)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import tensorflow as tf
>>> print(tf.__version__)
1.3.1
>>>
powerAI toolkit 에 포함된 tensorflow 로 동작
u0018670@sys-90257:~$ . /opt/DL/tensorflow/bin/tensorflow-activate
u0018670@sys-90257:~$ env | grep PATH
LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64:/usr/local/cuda-8.0/extras/CUPTI/lib64:/opt/DL/tensorflow/lib
PATH=/home/u0018670/bin:/home/u0018670/.local/bin:/home/u0018670/anaconda2/bin:/opt/ibm/xlC/current/bin:/opt/ibm/xlf/current/bin:/opt/at10.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/opt/DL/tensorflow/bin
PYTHONPATH=/opt/DL/tensorflow/lib/python2.7/site-packages
u0018670@sys-90257:~$ python
Python 2.7.13 |Anaconda, Inc.| (default, Sep 15 2017, 20:54:27)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import tensorflow as tf
>>> print(tf.__version__)
1.1.0
>>>
python system path 확인
u0018670@sys-90257:~$ python3
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/lib/python35.zip', '/usr/lib/python3.5', '/usr/lib/python3.5/plat-powerpc64le-linux-gnu', '/usr/lib/python3.5/lib-dynload', '/home/u0018670/.local/lib/python3.5/site-packages', '/usr/local/lib/python3.5/dist-packages', '/usr/lib/python3/dist-packages']
>>>
native python3 설치 후 link 생성
native python3 를 다음과 같이 설치 후,
u0018670@sys-90257:~$ sudo apt-get install python3
jupyter 도 pip install 설치
u0018670@sys-90254:~$ pip list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
bleach (2.1.1)
decorator (4.1.2)
entrypoints (0.2.3)
html5lib (1.0b10)
ipykernel (4.6.1)
ipython (6.2.1)
ipython-genutils (0.2.0)
ipywidgets (7.0.5)
jedi (0.11.0)
Jinja2 (2.10)
jsonschema (2.6.0)
jupyter (1.0.0)
jupyter-client (5.1.0)
jupyter-console (5.2.0)
jupyter-core (4.4.0)
MarkupSafe (1.0)
그런데 jupyter 가 실행이 안됨.
u0018670@sys-90257:~$ jupyter
-bash: /home/u0018670/.local/bin/jupyter: /usr/bin/python: bad interpreter: No such file or directory
안되는 이유는 /usr/bin/python 파일이 없어서 그래서 link 시켜주면 잘 됨.
u0018670@sys-90257:~$ sudo ln -s /usr/bin/python3 /usr/bin/python
u0018670@sys-90257:~$ python
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
u0018670@sys-90257:~$ jupyter
usage: jupyter [-h] [--version] [--config-dir] [--data-dir] [--runtime-dir]
[--paths] [--json]
[subcommand]
theano 기본 테스트
root@3498b2e65eb1:/tmp# python
Python 3.5.4 |Anaconda, Inc.| (default, Sep 19 2017, 14:42:07)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import theano
>>> import theano.tensor as T
>>> import numpy as np
>>> x = T.dscalar('x')
>>> y = x ** 2
>>> f = theano.function(inputs=[x], outputs=y)
>>> f(1)
array(1.0)
>>> f(2)
array(4.0)
>>> f(3)
array(9.0)
>>>
jupyter notebook 에서는 다음과 같이 출력 확인
anaconda 를 통해 pip install 로 설치된 tensorflow 1.3.1 로 동작 확인
u0018670@sys-90257:~$ env | grep PATH
PATH=/home/u0018670/bin:/home/u0018670/.local/bin:/home/u0018670/anaconda2/bin:/opt/ibm/xlC/current/bin:/opt/ibm/xlf/current/bin:/opt/at10.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
u0018670@sys-90257:~$ python
Python 2.7.13 |Anaconda, Inc.| (default, Sep 15 2017, 20:54:27)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import tensorflow as tf
>>> print(tf.__version__)
1.3.1
>>>
powerAI toolkit 에 포함된 tensorflow 로 동작
u0018670@sys-90257:~$ . /opt/DL/tensorflow/bin/tensorflow-activate
u0018670@sys-90257:~$ env | grep PATH
LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64:/usr/local/cuda-8.0/extras/CUPTI/lib64:/opt/DL/tensorflow/lib
PATH=/home/u0018670/bin:/home/u0018670/.local/bin:/home/u0018670/anaconda2/bin:/opt/ibm/xlC/current/bin:/opt/ibm/xlf/current/bin:/opt/at10.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/opt/DL/tensorflow/bin
PYTHONPATH=/opt/DL/tensorflow/lib/python2.7/site-packages
u0018670@sys-90257:~$ python
Python 2.7.13 |Anaconda, Inc.| (default, Sep 15 2017, 20:54:27)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import tensorflow as tf
>>> print(tf.__version__)
1.1.0
>>>
python system path 확인
u0018670@sys-90257:~$ python3
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/lib/python35.zip', '/usr/lib/python3.5', '/usr/lib/python3.5/plat-powerpc64le-linux-gnu', '/usr/lib/python3.5/lib-dynload', '/home/u0018670/.local/lib/python3.5/site-packages', '/usr/local/lib/python3.5/dist-packages', '/usr/lib/python3/dist-packages']
>>>
native python3 설치 후 link 생성
native python3 를 다음과 같이 설치 후,
u0018670@sys-90257:~$ sudo apt-get install python3
jupyter 도 pip install 설치
u0018670@sys-90254:~$ pip list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
bleach (2.1.1)
decorator (4.1.2)
entrypoints (0.2.3)
html5lib (1.0b10)
ipykernel (4.6.1)
ipython (6.2.1)
ipython-genutils (0.2.0)
ipywidgets (7.0.5)
jedi (0.11.0)
Jinja2 (2.10)
jsonschema (2.6.0)
jupyter (1.0.0)
jupyter-client (5.1.0)
jupyter-console (5.2.0)
jupyter-core (4.4.0)
MarkupSafe (1.0)
그런데 jupyter 가 실행이 안됨.
u0018670@sys-90257:~$ jupyter
-bash: /home/u0018670/.local/bin/jupyter: /usr/bin/python: bad interpreter: No such file or directory
안되는 이유는 /usr/bin/python 파일이 없어서 그래서 link 시켜주면 잘 됨.
u0018670@sys-90257:~$ sudo ln -s /usr/bin/python3 /usr/bin/python
u0018670@sys-90257:~$ python
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
u0018670@sys-90257:~$ jupyter
usage: jupyter [-h] [--version] [--config-dir] [--data-dir] [--runtime-dir]
[--paths] [--json]
[subcommand]
theano 기본 테스트
root@3498b2e65eb1:/tmp# python
Python 3.5.4 |Anaconda, Inc.| (default, Sep 19 2017, 14:42:07)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import theano
>>> import theano.tensor as T
>>> import numpy as np
>>> x = T.dscalar('x')
>>> y = x ** 2
>>> f = theano.function(inputs=[x], outputs=y)
>>> f(1)
array(1.0)
>>> f(2)
array(4.0)
>>> f(3)
array(9.0)
>>>
jupyter notebook 에서는 다음과 같이 출력 확인
2017년 12월 6일 수요일
내가 자주쓰는, 기억해야 하는 command - Ubuntu
환경변수 확인
u0018670@sys-90254:~$ more /etc/environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
u0018670@sys-90254:~$ env | grep -i path
PATH=/home/u0018670/bin:/home/u0018670/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
설치된 패키지 경로 확인
u0018670@sys-90254:~$ dpkg -S python-pip-whl
python-pip-whl: /usr/share/doc/python-pip-whl/changelog.Debian.gz
python-pip-whl: /usr/share/doc/python-pip-whl/copyright
python-pip-whl: /usr/share/doc/python-pip-whl
u0018670@sys-90254:~$ dpkg -S ifconfig
net-tools: /sbin/ifconfig
net-tools: /usr/share/man/de/man8/ifconfig.8.gz
linux-headers-4.4.0-103: /usr/src/linux-headers-4.4.0-103/tools/hv/hv_set_ifconfig.sh
net-tools: /usr/share/man/man8/ifconfig.8.gz
net-tools: /usr/share/man/pt_BR/man8/ifconfig.8.gz
linux-headers-4.4.0-104: /usr/src/linux-headers-4.4.0-104/tools/hv/hv_set_ifconfig.sh
net-tools: /usr/share/man/fr/man8/ifconfig.8.gz
/opt/at10.0 경로
IBM 패키지 중 advance-toolchain, xlc 등이 설치되는 경로 여기에 pip3 도 있고 gcc 도 있고... 과감히 삭제
u0018670@sys-90257:/opt/at10.0/bin$ dpkg -l |grep advance
ii advance-toolchain-at10.0-devel 10.0-2 ppc64el Advance Toolchain
ii advance-toolchain-at10.0-mcore-libs 10.0-2 ppc64el Advance Toolchain
ii advance-toolchain-at10.0-perf 10.0-2 ppc64el Advance Toolchain
ii advance-toolchain-at10.0-runtime 10.0-2 ppc64el Advance Toolchain
u0018670@sys-90254:~$ more /etc/environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
u0018670@sys-90254:~$ env | grep -i path
PATH=/home/u0018670/bin:/home/u0018670/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
설치된 패키지 경로 확인
u0018670@sys-90254:~$ dpkg -S python-pip-whl
python-pip-whl: /usr/share/doc/python-pip-whl/changelog.Debian.gz
python-pip-whl: /usr/share/doc/python-pip-whl/copyright
python-pip-whl: /usr/share/doc/python-pip-whl
u0018670@sys-90254:~$ dpkg -S ifconfig
net-tools: /sbin/ifconfig
net-tools: /usr/share/man/de/man8/ifconfig.8.gz
linux-headers-4.4.0-103: /usr/src/linux-headers-4.4.0-103/tools/hv/hv_set_ifconfig.sh
net-tools: /usr/share/man/man8/ifconfig.8.gz
net-tools: /usr/share/man/pt_BR/man8/ifconfig.8.gz
linux-headers-4.4.0-104: /usr/src/linux-headers-4.4.0-104/tools/hv/hv_set_ifconfig.sh
net-tools: /usr/share/man/fr/man8/ifconfig.8.gz
/opt/at10.0 경로
IBM 패키지 중 advance-toolchain, xlc 등이 설치되는 경로 여기에 pip3 도 있고 gcc 도 있고... 과감히 삭제
u0018670@sys-90257:/opt/at10.0/bin$ dpkg -l |grep advance
ii advance-toolchain-at10.0-devel 10.0-2 ppc64el Advance Toolchain
ii advance-toolchain-at10.0-mcore-libs 10.0-2 ppc64el Advance Toolchain
ii advance-toolchain-at10.0-perf 10.0-2 ppc64el Advance Toolchain
ii advance-toolchain-at10.0-runtime 10.0-2 ppc64el Advance Toolchain
2017년 12월 5일 화요일
Python 2.7 과 Python 3.5 동시 사용 - Ubuntu
Anaconda2 로 python 2.7 이 설치된 환경에서 root 에 영향이 없이, 다시말해 Anaconda3 를 설치하지 않아도 python 3.5 를 사용하기 위해 다음과 같이 수행
현재 설치된 anaconda2 확인
u0018670@sys-90257:~$ which python
/home/u0018670/anaconda2/bin/python
jupyter kernal 확인
u0018670@sys-90257:~$ jupyter kernelspec list
Available kernels:
python2 /home/u0018670/anaconda2/share/jupyter/kernels/python2
jupyter notebook 을 실행해봐도 Python2 만 확인
u0018670@sys-90257:~$ conda update conda
Fetching package metadata ...........
Solving package specifications: .
# All requested packages already installed.
# packages in environment at /home/u0018670/anaconda2:
#
conda 4.3.27 py27_0
python 3.5 환경으로 업데이트
u0018670@sys-90257:~$ conda create -n py35 python=3.5 anaconda
Fetching package metadata ...........
Solving package specifications: .
Package plan for installation in environment /home/u0018670/anaconda2/envs/py35:
The following NEW packages will be INSTALLED:
alabaster: 0.7.10-py35_0
anaconda: 5.0.0-py35h419c926_0
anaconda-client: 1.6.3-py35_0
anaconda-project: 0.6.0-py35_0
asn1crypto: 0.22.0-py35_0
astroid: 1.5.3-py35_0
astropy: 2.0.1-np113py35_1
babel: 2.5.0-py35_0
backports: 1.0-py35_0
beautifulsoup4: 4.6.0-py35_0
bitarray: 0.8.1-py35_0
......................
Proceed ([y]/n)? y
python-3.5.4-0 100% |#####################################################################################| Time: 0:00:01 14.46 MB/s
alabaster-0.7. 100% |#####################################################################################| Time: 0:00:00 15.39 MB/s
asn1crypto-0.2 100% |#####################################################################################| Time: 0:00:00 16.71 MB/s
backports-1.0- 100% |#####################################################################################| Time: 0:00:00 3.98 MB/s
beautifulsoup4 100% |#####################################################################################| Time: 0:00:00 18.22 MB/s
bitarray-0.8.1 100% |#####################################################################################| Time: 0:00:00 25.68 MB/s
boto-2.48.0-py 100% |#####################################################################################| Time: 0:00:00 3.83 MB/s
......................
anaconda-5.0.0 100% |#####################################################################################| Time: 0:00:00 12.86 MB/s
#
# To activate this environment, use:
# > source activate py35
#
# To deactivate an active environment, use:
# > source deactivate
#
u0018670@sys-90257:~$ source activate py35
(py35) u0018670@sys-90257:~$ jupyter notebook
(py35) u0018670@sys-90257:~/anaconda2$ which python
/home/u0018670/anaconda2/envs/py35/bin/python
(py35) u0018670@sys-90257:~/anaconda2$ source deactivate
u0018670@sys-90257:~/anaconda2$
추가적으로, 특정 버전의 python 패키지를 설치할때는 다음과 같이
그냥 keras 설치를 해버리면 최신 버전이 설치됨.
nimbix@JARVICENAE-0A0A183D:~$ pip install keras
Collecting keras
Downloading Keras-2.1.2-py2.py3-none-any.whl (304kB)
100% |████████████████████████████████| 307kB 4.3MB/s
Requirement already satisfied: scipy>=0.14 in ./anaconda3/lib/python3.6/site-packages (from keras)
Requirement already satisfied: six>=1.9.0 in ./anaconda3/lib/python3.6/site-packages (from keras)
Requirement already satisfied: pyyaml in ./anaconda3/lib/python3.6/site-packages (from keras)
Requirement already satisfied: numpy>=1.9.1 in ./anaconda3/lib/python3.6/site-packages (from keras)
Installing collected packages: keras
Successfully installed keras-2.1.2
그런데,
nimbix@JARVICENAE-0A0A183D:~$ pip install keras==1.2
Collecting keras==1.2
Downloading Keras-1.2.0.tar.gz (167kB)
100% |████████████████████████████████| 174kB 6.4MB/s
Collecting theano (from keras==1.2)
Downloading Theano-1.0.1.tar.gz (2.8MB)
100% |████████████████████████████████| 2.8MB 506kB/s
Requirement already satisfied: pyyaml in ./anaconda3/lib/python3.6/site-packages (from keras==1.2)
Requirement already satisfied: six in ./anaconda3/lib/python3.6/site-packages (from keras==1.2)
Requirement already satisfied: numpy>=1.9.1 in ./anaconda3/lib/python3.6/site-packages (from theano->keras==1.2)
Requirement already satisfied: scipy>=0.14 in ./anaconda3/lib/python3.6/site-packages (from theano->keras==1.2)
Building wheels for collected packages: keras, theano
Running setup.py bdist_wheel for keras ... done
Stored in directory: /home/nimbix/.cache/pip/wheels/f8/2c/8e/27d952128220b1acf6fcd5e692269c6e8d60744c15fd160c2c
Running setup.py bdist_wheel for theano ... done
Stored in directory: /home/nimbix/.cache/pip/wheels/46/a2/7d/b4cac381d5151daa9f9e0b3e4e4b65edaea6355ae296c97cf2
Successfully built keras theano
Installing collected packages: theano, keras
Found existing installation: Keras 2.1.2
Uninstalling Keras-2.1.2:
Successfully uninstalled Keras-2.1.2
Successfully installed keras-1.2.0 theano-1.0.1
기존 2.1.2 를 자동으로 삭제하고 1.2 가 설치됨.
현재 설치된 anaconda2 확인
u0018670@sys-90257:~$ which python
/home/u0018670/anaconda2/bin/python
jupyter kernal 확인
u0018670@sys-90257:~$ jupyter kernelspec list
Available kernels:
python2 /home/u0018670/anaconda2/share/jupyter/kernels/python2
jupyter notebook 을 실행해봐도 Python2 만 확인
u0018670@sys-90257:~$ conda update conda
Fetching package metadata ...........
Solving package specifications: .
# All requested packages already installed.
# packages in environment at /home/u0018670/anaconda2:
#
conda 4.3.27 py27_0
python 3.5 환경으로 업데이트
u0018670@sys-90257:~$ conda create -n py35 python=3.5 anaconda
Fetching package metadata ...........
Solving package specifications: .
Package plan for installation in environment /home/u0018670/anaconda2/envs/py35:
The following NEW packages will be INSTALLED:
alabaster: 0.7.10-py35_0
anaconda: 5.0.0-py35h419c926_0
anaconda-client: 1.6.3-py35_0
anaconda-project: 0.6.0-py35_0
asn1crypto: 0.22.0-py35_0
astroid: 1.5.3-py35_0
astropy: 2.0.1-np113py35_1
babel: 2.5.0-py35_0
backports: 1.0-py35_0
beautifulsoup4: 4.6.0-py35_0
bitarray: 0.8.1-py35_0
......................
Proceed ([y]/n)? y
python-3.5.4-0 100% |#####################################################################################| Time: 0:00:01 14.46 MB/s
alabaster-0.7. 100% |#####################################################################################| Time: 0:00:00 15.39 MB/s
asn1crypto-0.2 100% |#####################################################################################| Time: 0:00:00 16.71 MB/s
backports-1.0- 100% |#####################################################################################| Time: 0:00:00 3.98 MB/s
beautifulsoup4 100% |#####################################################################################| Time: 0:00:00 18.22 MB/s
bitarray-0.8.1 100% |#####################################################################################| Time: 0:00:00 25.68 MB/s
boto-2.48.0-py 100% |#####################################################################################| Time: 0:00:00 3.83 MB/s
......................
anaconda-5.0.0 100% |#####################################################################################| Time: 0:00:00 12.86 MB/s
#
# To activate this environment, use:
# > source activate py35
#
# To deactivate an active environment, use:
# > source deactivate
#
u0018670@sys-90257:~$ source activate py35
(py35) u0018670@sys-90257:~$ jupyter notebook
(py35) u0018670@sys-90257:~/anaconda2$ which python
/home/u0018670/anaconda2/envs/py35/bin/python
(py35) u0018670@sys-90257:~/anaconda2$ source deactivate
u0018670@sys-90257:~/anaconda2$
추가적으로, 특정 버전의 python 패키지를 설치할때는 다음과 같이
그냥 keras 설치를 해버리면 최신 버전이 설치됨.
nimbix@JARVICENAE-0A0A183D:~$ pip install keras
Collecting keras
Downloading Keras-2.1.2-py2.py3-none-any.whl (304kB)
100% |████████████████████████████████| 307kB 4.3MB/s
Requirement already satisfied: scipy>=0.14 in ./anaconda3/lib/python3.6/site-packages (from keras)
Requirement already satisfied: six>=1.9.0 in ./anaconda3/lib/python3.6/site-packages (from keras)
Requirement already satisfied: pyyaml in ./anaconda3/lib/python3.6/site-packages (from keras)
Requirement already satisfied: numpy>=1.9.1 in ./anaconda3/lib/python3.6/site-packages (from keras)
Installing collected packages: keras
Successfully installed keras-2.1.2
그런데,
nimbix@JARVICENAE-0A0A183D:~$ pip install keras==1.2
Collecting keras==1.2
Downloading Keras-1.2.0.tar.gz (167kB)
100% |████████████████████████████████| 174kB 6.4MB/s
Collecting theano (from keras==1.2)
Downloading Theano-1.0.1.tar.gz (2.8MB)
100% |████████████████████████████████| 2.8MB 506kB/s
Requirement already satisfied: pyyaml in ./anaconda3/lib/python3.6/site-packages (from keras==1.2)
Requirement already satisfied: six in ./anaconda3/lib/python3.6/site-packages (from keras==1.2)
Requirement already satisfied: numpy>=1.9.1 in ./anaconda3/lib/python3.6/site-packages (from theano->keras==1.2)
Requirement already satisfied: scipy>=0.14 in ./anaconda3/lib/python3.6/site-packages (from theano->keras==1.2)
Building wheels for collected packages: keras, theano
Running setup.py bdist_wheel for keras ... done
Stored in directory: /home/nimbix/.cache/pip/wheels/f8/2c/8e/27d952128220b1acf6fcd5e692269c6e8d60744c15fd160c2c
Running setup.py bdist_wheel for theano ... done
Stored in directory: /home/nimbix/.cache/pip/wheels/46/a2/7d/b4cac381d5151daa9f9e0b3e4e4b65edaea6355ae296c97cf2
Successfully built keras theano
Installing collected packages: theano, keras
Found existing installation: Keras 2.1.2
Uninstalling Keras-2.1.2:
Successfully uninstalled Keras-2.1.2
Successfully installed keras-1.2.0 theano-1.0.1
기존 2.1.2 를 자동으로 삭제하고 1.2 가 설치됨.
2017년 12월 4일 월요일
폐쇄망에서 python 패키지 설치 - Ubuntu
먼저, 인터넷이 되는 곳에서 원하는 패키지를 다음과 같이 다운로드 받는다.
online@ubuntu:~$ pip install --download . tensorflow
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting tensorflow
Downloading tensorflow-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl (40.7MB)
100% |████████████████████████████████| 40.8MB 23kB/s
Saved ./tensorflow-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting enum34>=1.1.6 (from tensorflow)
Downloading enum34-1.1.6-py2-none-any.whl
Saved ./enum34-1.1.6-py2-none-any.whl
Collecting backports.weakref>=1.0rc1 (from tensorflow)
Downloading backports.weakref-1.0.post1-py2.py3-none-any.whl
Saved ./backports.weakref-1.0.post1-py2.py3-none-any.whl
Collecting wheel (from tensorflow)
Downloading wheel-0.30.0-py2.py3-none-any.whl (49kB)
100% |████████████████████████████████| 51kB 6.9MB/s
Saved ./wheel-0.30.0-py2.py3-none-any.whl
Collecting mock>=2.0.0 (from tensorflow)
Downloading mock-2.0.0-py2.py3-none-any.whl (56kB)
100% |████████████████████████████████| 61kB 2.7MB/s
Saved ./mock-2.0.0-py2.py3-none-any.whl
Collecting tensorflow-tensorboard<0.5.0,>=0.4.0rc1 (from tensorflow)
Downloading tensorflow_tensorboard-0.4.0rc3-py2-none-any.whl (1.7MB)
100% |████████████████████████████████| 1.7MB 537kB/s
Saved ./tensorflow_tensorboard-0.4.0rc3-py2-none-any.whl
Collecting numpy>=1.12.1 (from tensorflow)
Downloading numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl (16.6MB)
100% |████████████████████████████████| 16.7MB 66kB/s
Saved ./numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Collecting protobuf>=3.3.0 (from tensorflow)
Downloading protobuf-3.5.0.post1-cp27-cp27mu-manylinux1_x86_64.whl (6.4MB)
100% |████████████████████████████████| 6.4MB 166kB/s
Saved ./protobuf-3.5.0.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting six>=1.10.0 (from tensorflow)
Downloading six-1.11.0-py2.py3-none-any.whl
Saved ./six-1.11.0-py2.py3-none-any.whl
Collecting funcsigs>=1; python_version < "3.3" (from mock>=2.0.0->tensorflow)
Downloading funcsigs-1.0.2-py2.py3-none-any.whl
Saved ./funcsigs-1.0.2-py2.py3-none-any.whl
Collecting pbr>=0.11 (from mock>=2.0.0->tensorflow)
Downloading pbr-3.1.1-py2.py3-none-any.whl (99kB)
100% |████████████████████████████████| 102kB 1.6MB/s
Saved ./pbr-3.1.1-py2.py3-none-any.whl
Collecting bleach==1.5.0 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Downloading bleach-1.5.0-py2.py3-none-any.whl
Saved ./bleach-1.5.0-py2.py3-none-any.whl
Collecting futures>=3.1.1; python_version < "3.2" (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Downloading futures-3.2.0-py2-none-any.whl
Saved ./futures-3.2.0-py2-none-any.whl
Collecting markdown>=2.6.8 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Downloading Markdown-2.6.9.tar.gz (271kB)
100% |████████████████████████████████| 276kB 1.3MB/s
Saved ./Markdown-2.6.9.tar.gz
Collecting werkzeug>=0.11.10 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Downloading Werkzeug-0.12.2-py2.py3-none-any.whl (312kB)
100% |████████████████████████████████| 317kB 1.3MB/s
Saved ./Werkzeug-0.12.2-py2.py3-none-any.whl
Collecting html5lib==0.9999999 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Downloading html5lib-0.9999999.tar.gz (889kB)
100% |████████████████████████████████| 890kB 744kB/s
Saved ./html5lib-0.9999999.tar.gz
Collecting setuptools (from protobuf>=3.3.0->tensorflow)
Downloading setuptools-38.2.4-py2.py3-none-any.whl (489kB)
100% |████████████████████████████████| 491kB 1.0MB/s
Saved ./setuptools-38.2.4-py2.py3-none-any.whl
Successfully downloaded tensorflow enum34 backports.weakref wheel mock tensorflow-tensorboard numpy protobuf six funcsigs pbr bleach futures markdown werkzeug html5lib setuptools
다운로드 받은 패키지들은 의존성에 의해 관련된 패키지들도 함께 다운받아지며 이것들을 폐쇄망 서버로 이동(tensorflow 디렉토리) 후, 다음과 같이 설치.
offline@ubuntu:~/tensorflow$ pip install --no-index --find-links=. tensorflow
Collecting tensorflow
Requirement already satisfied: enum34>=1.1.6 in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow)
Collecting backports.weakref>=1.0rc1 (from tensorflow)
Requirement already satisfied: wheel in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow)
Collecting mock>=2.0.0 (from tensorflow)
Collecting tensorflow-tensorboard<0.5.0,>=0.4.0rc1 (from tensorflow)
Requirement already satisfied: numpy>=1.12.1 in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow)
Collecting protobuf>=3.3.0 (from tensorflow)
Requirement already satisfied: six>=1.10.0 in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/testuser01/anaconda2/lib/python2.7/site-packages (from mock>=2.0.0->tensorflow)
Collecting pbr>=0.11 (from mock>=2.0.0->tensorflow)
Collecting bleach==1.5.0 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Requirement already satisfied: futures>=3.1.1; python_version < "3.2" in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Collecting markdown>=2.6.8 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Requirement already satisfied: werkzeug>=0.11.10 in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Collecting html5lib==0.9999999 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Requirement already satisfied: setuptools in /home/testuser01/anaconda2/lib/python2.7/site-packages (from protobuf>=3.3.0->tensorflow)
Building wheels for collected packages: markdown, html5lib
Running setup.py bdist_wheel for markdown ... done
Stored in directory: /home/testuser01/.cache/pip/wheels/51/36/90/47aaba211d675deb9af500b7a894541103b283c5f18e0ef529
Running setup.py bdist_wheel for html5lib ... done
Stored in directory: /home/testuser01/.cache/pip/wheels/22/7a/d3/e96250b678c5b3af1d50336781a06e05568d81251c10440b4a
Successfully built markdown html5lib
Installing collected packages: backports.weakref, pbr, mock, html5lib, bleach, protobuf, markdown, tensorflow-tensorboard, tensorflow
Found existing installation: html5lib 0.999999999
Uninstalling html5lib-0.999999999:
Successfully uninstalled html5lib-0.999999999
Found existing installation: bleach 2.0.0
Uninstalling bleach-2.0.0:
Successfully uninstalled bleach-2.0.0
Successfully installed backports.weakref-1.0.post1 bleach-1.5.0 html5lib-0.9999999 markdown-2.6.9 mock-2.0.0 pbr-3.1.1 protobuf-3.5.0.post1 tensorflow-1.4.0 tensorflow-tensorboard-0.4.0rc3
offline@ubuntu:~/tensorflow$ conda list | grep tensorflow
tensorflow 1.4.0 <pip>
tensorflow-tensorboard 0.4.0rc3 <pip>
online@ubuntu:~$ pip install --download . tensorflow
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting tensorflow
Downloading tensorflow-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl (40.7MB)
100% |████████████████████████████████| 40.8MB 23kB/s
Saved ./tensorflow-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting enum34>=1.1.6 (from tensorflow)
Downloading enum34-1.1.6-py2-none-any.whl
Saved ./enum34-1.1.6-py2-none-any.whl
Collecting backports.weakref>=1.0rc1 (from tensorflow)
Downloading backports.weakref-1.0.post1-py2.py3-none-any.whl
Saved ./backports.weakref-1.0.post1-py2.py3-none-any.whl
Collecting wheel (from tensorflow)
Downloading wheel-0.30.0-py2.py3-none-any.whl (49kB)
100% |████████████████████████████████| 51kB 6.9MB/s
Saved ./wheel-0.30.0-py2.py3-none-any.whl
Collecting mock>=2.0.0 (from tensorflow)
Downloading mock-2.0.0-py2.py3-none-any.whl (56kB)
100% |████████████████████████████████| 61kB 2.7MB/s
Saved ./mock-2.0.0-py2.py3-none-any.whl
Collecting tensorflow-tensorboard<0.5.0,>=0.4.0rc1 (from tensorflow)
Downloading tensorflow_tensorboard-0.4.0rc3-py2-none-any.whl (1.7MB)
100% |████████████████████████████████| 1.7MB 537kB/s
Saved ./tensorflow_tensorboard-0.4.0rc3-py2-none-any.whl
Collecting numpy>=1.12.1 (from tensorflow)
Downloading numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl (16.6MB)
100% |████████████████████████████████| 16.7MB 66kB/s
Saved ./numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Collecting protobuf>=3.3.0 (from tensorflow)
Downloading protobuf-3.5.0.post1-cp27-cp27mu-manylinux1_x86_64.whl (6.4MB)
100% |████████████████████████████████| 6.4MB 166kB/s
Saved ./protobuf-3.5.0.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting six>=1.10.0 (from tensorflow)
Downloading six-1.11.0-py2.py3-none-any.whl
Saved ./six-1.11.0-py2.py3-none-any.whl
Collecting funcsigs>=1; python_version < "3.3" (from mock>=2.0.0->tensorflow)
Downloading funcsigs-1.0.2-py2.py3-none-any.whl
Saved ./funcsigs-1.0.2-py2.py3-none-any.whl
Collecting pbr>=0.11 (from mock>=2.0.0->tensorflow)
Downloading pbr-3.1.1-py2.py3-none-any.whl (99kB)
100% |████████████████████████████████| 102kB 1.6MB/s
Saved ./pbr-3.1.1-py2.py3-none-any.whl
Collecting bleach==1.5.0 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Downloading bleach-1.5.0-py2.py3-none-any.whl
Saved ./bleach-1.5.0-py2.py3-none-any.whl
Collecting futures>=3.1.1; python_version < "3.2" (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Downloading futures-3.2.0-py2-none-any.whl
Saved ./futures-3.2.0-py2-none-any.whl
Collecting markdown>=2.6.8 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Downloading Markdown-2.6.9.tar.gz (271kB)
100% |████████████████████████████████| 276kB 1.3MB/s
Saved ./Markdown-2.6.9.tar.gz
Collecting werkzeug>=0.11.10 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Downloading Werkzeug-0.12.2-py2.py3-none-any.whl (312kB)
100% |████████████████████████████████| 317kB 1.3MB/s
Saved ./Werkzeug-0.12.2-py2.py3-none-any.whl
Collecting html5lib==0.9999999 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Downloading html5lib-0.9999999.tar.gz (889kB)
100% |████████████████████████████████| 890kB 744kB/s
Saved ./html5lib-0.9999999.tar.gz
Collecting setuptools (from protobuf>=3.3.0->tensorflow)
Downloading setuptools-38.2.4-py2.py3-none-any.whl (489kB)
100% |████████████████████████████████| 491kB 1.0MB/s
Saved ./setuptools-38.2.4-py2.py3-none-any.whl
Successfully downloaded tensorflow enum34 backports.weakref wheel mock tensorflow-tensorboard numpy protobuf six funcsigs pbr bleach futures markdown werkzeug html5lib setuptools
다운로드 받은 패키지들은 의존성에 의해 관련된 패키지들도 함께 다운받아지며 이것들을 폐쇄망 서버로 이동(tensorflow 디렉토리) 후, 다음과 같이 설치.
offline@ubuntu:~/tensorflow$ pip install --no-index --find-links=. tensorflow
Collecting tensorflow
Requirement already satisfied: enum34>=1.1.6 in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow)
Collecting backports.weakref>=1.0rc1 (from tensorflow)
Requirement already satisfied: wheel in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow)
Collecting mock>=2.0.0 (from tensorflow)
Collecting tensorflow-tensorboard<0.5.0,>=0.4.0rc1 (from tensorflow)
Requirement already satisfied: numpy>=1.12.1 in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow)
Collecting protobuf>=3.3.0 (from tensorflow)
Requirement already satisfied: six>=1.10.0 in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/testuser01/anaconda2/lib/python2.7/site-packages (from mock>=2.0.0->tensorflow)
Collecting pbr>=0.11 (from mock>=2.0.0->tensorflow)
Collecting bleach==1.5.0 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Requirement already satisfied: futures>=3.1.1; python_version < "3.2" in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Collecting markdown>=2.6.8 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Requirement already satisfied: werkzeug>=0.11.10 in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Collecting html5lib==0.9999999 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Requirement already satisfied: setuptools in /home/testuser01/anaconda2/lib/python2.7/site-packages (from protobuf>=3.3.0->tensorflow)
Building wheels for collected packages: markdown, html5lib
Running setup.py bdist_wheel for markdown ... done
Stored in directory: /home/testuser01/.cache/pip/wheels/51/36/90/47aaba211d675deb9af500b7a894541103b283c5f18e0ef529
Running setup.py bdist_wheel for html5lib ... done
Stored in directory: /home/testuser01/.cache/pip/wheels/22/7a/d3/e96250b678c5b3af1d50336781a06e05568d81251c10440b4a
Successfully built markdown html5lib
Installing collected packages: backports.weakref, pbr, mock, html5lib, bleach, protobuf, markdown, tensorflow-tensorboard, tensorflow
Found existing installation: html5lib 0.999999999
Uninstalling html5lib-0.999999999:
Successfully uninstalled html5lib-0.999999999
Found existing installation: bleach 2.0.0
Uninstalling bleach-2.0.0:
Successfully uninstalled bleach-2.0.0
Successfully installed backports.weakref-1.0.post1 bleach-1.5.0 html5lib-0.9999999 markdown-2.6.9 mock-2.0.0 pbr-3.1.1 protobuf-3.5.0.post1 tensorflow-1.4.0 tensorflow-tensorboard-0.4.0rc3
offline@ubuntu:~/tensorflow$ conda list | grep tensorflow
tensorflow 1.4.0 <pip>
tensorflow-tensorboard 0.4.0rc3 <pip>
피드 구독하기:
글 (Atom)