이 블로그 검색

2018년 1월 22일 월요일

cifar10 돌려보기

github (https://github.com/tensorflow/models/tree/master/tutorials/image/cifar10) 에서 cifar10.py, cifar10_input.py, cifar10_train 를 다운 받은 후 cifar10_train.py 를 수행

2017년 12월 18일 월요일

docker 기본 사용 - Ubuntu

설치된 docker engine 확인
u0018670@sys-90572:~$ dpkg -l |grep docker
ii  docker.io                           1.13.1-0ubuntu1~16.04.2                    ppc64el      Linux container runtime
ii  runc                                1.0.0~rc2+docker1.13.1-0ubuntu1~16.04.1    ppc64el      Open Container Project - runtime


아무런 docker image 가 없는 것을 확인
u0018670@sys-90572:~$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE


그래서, ppc64le/ubuntu docker image 를 실행시키면 latest 로 docker hub 로 부터 docker tag 가 latest 인 것을 download 후 docker 실행
u0018670@sys-90572:~$ sudo docker run -it ppc64le/ubuntu
Unable to find image 'ppc64le/ubuntu:latest' locally
latest: Pulling from ppc64le/ubuntu
586a07d977d9: Pull complete
2bf911559ec0: Pull complete
48999c060756: Pull complete
fa5e5494889f: Pull complete
3bd4f80d03d3: Pull complete
Digest: sha256:79d11934f87adb78614d5ca93092f6681ca154c2bd9df56a7027615c56d53e41
Status: Downloaded newer image for ppc64le/ubuntu:latest

root@b916c38bbbb7:/# ls     <== docker container 안
bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
root@b916c38bbbb7:/# exit

exit

생성된 docker image 확인
u0018670@sys-90572:~$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
ppc64le/ubuntu      latest              44c2eaa37868        3 days ago          150 MB


수행중인 container 확인
u0018670@sys-90572:~$ sudo docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                      PORTS               NAMES
b916c38bbbb7        ppc64le/ubuntu      "/bin/bash"         54 seconds ago      Exited (0) 17 seconds ago                       epic_cori


exit 한 container 재시작
u0018670@sys-90727:~$ sudo docker start confident_euclid
confident_euclid
u0018670@sys-90727:~$ sudo docker attach confident_euclid
root@0f647ddac01a:/#


수행중인 container 를 image 로 commit
u0018670@sys-90727:~$ sudo docker commit 58d0d814ae31 ppc64le/ubuntu:latest
sha256:4de2af599de3a7a0aa2351a9fcb6e9b8cbaa8f3a345d6fed03291477097da82d
u0018670@sys-90727:~$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED              SIZE
ppc64le/ubuntu      latest              4de2af599de3        About a minute ago   260MB
ppc64le/ubuntu      16.04               44c2eaa37868        3 weeks ago          150MB


수행중인 container 삭제
u0018670@sys-90572:~$ sudo docker rm epic_cori
epic_cori

u0018670@sys-90572:~$ sudo docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

u0018670@sys-90572:~$
한꺼번에 모든 container 삭제는 docker rm $(docker ps -a -q)

다음은 2개의 container 가 시작된 상황. (ppc64le/ubuntu 이미지로 eager_hugle, quirky_pike)
u0018670@sys-90572:~$ sudo docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED              STATUS                      PORTS               NAMES
eaf515d4a2c6        ppc64le/ubuntu      "/bin/bash"         16 seconds ago       Exited (0) 12 seconds ago                       eager_hugle
4e3e7338019b        ppc64le/ubuntu      "/bin/bash"         About a minute ago   Exited (0) 21 seconds ago                       quirky_pike


container 수행 중이기에 삭제 되지 않는 docker image, rm 옵션으로 container 를 먼저 삭제 후 image 삭제
u0018670@sys-90572:~$ sudo docker rmi ppc64le/ubuntu
Error response from daemon: conflict: unable to remove repository reference "ppc64le/ubuntu" (must force) - container 4e3e7338019b is using its referenced image 44c2eaa37868
u0018670@sys-90572:~$


docker hub 에 생성된 이미지 upload
u0018670@sys-90461:~/data$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
leearmy/cudnn6      ppc64le             de632816acda        About an hour ago   6.58GB
leearmy/ubuntu      16.04               23296ffbcbfc        4 hours ago         359MB
ppc64le/ubuntu      16.04               44c2eaa37868        3 days ago          150MB

다음 명령어를 수행하기 위해서는 먼저 https://hub.docker.com/ 에 계정과 repository 를 생성하여야 함
u0018670@sys-90461:~/data$ sudo docker login --username=leearmy
Password:
Login Succeeded
u0018670@sys-90461:~/data$ sudo docker push leearmy/cudnn6:ppc64le
The push refers to a repository [docker.io/leearmy/cudnn6]
026ec1f6d915: Pushed
c963a10dda0d: Pushed
4a08cdc31ed0: Pushed
49076433d00c: Pushed
8788fee75149: Pushed
d9561c8b4bf3: Pushed
54ac221998f0: Pushed
b10475677e3b: Pushed
adf3a2526761: Pushed
e8449de7ace1: Pushed
0c5b7df1df7c: Pushed
b1dc141f6073: Mounted from ppc64le/ubuntu
448bb1e132fc: Mounted from ppc64le/ubuntu
e7f4651b678d: Mounted from ppc64le/ubuntu
ee8f1c150908: Mounted from ppc64le/ubuntu
258cc4d5502c: Mounted from ppc64le/ubuntu
ppc64le: digest: sha256:3df7ee55f48c08fe7dc836c60053867dac28ada8b7eb9b9413b09e5361e4937c size: 3690


다음과 같이 https://hub.docker.com/ 에서 올려진 이미지 확인


docker hub 에 올려진 이미지는 다른 서버에서 pull
u0018670@sys-90572:~$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
ppc64le/ubuntu      latest              44c2eaa37868        3 days ago          150 MB
u0018670@sys-90572:~$ sudo docker pull leearmy/cudnn6:ppc64le
ppc64le: Pulling from leearmy/cudnn6
586a07d977d9: Already exists
2bf911559ec0: Already exists
48999c060756: Already exists
fa5e5494889f: Already exists
3bd4f80d03d3: Already exists
3d482c0c4d09: Pull complete
9e891e52327a: Pull complete
a40daf5d9d7f: Pull complete
2ca681ce2118: Pull complete
111379f9235b: Pull complete
9db8b0fb9301: Pull complete
b2b39482e9b8: Pull complete
93f5e3ec7d59: Pull complete
38e0ece0a1d4: Pull complete
dbc7427ab4ea: Pull complete
f1cc2b939ac9: Pull complete
Digest: sha256:3df7ee55f48c08fe7dc836c60053867dac28ada8b7eb9b9413b09e5361e4937c
Status: Downloaded newer image for leearmy/cudnn6:ppc64le
u0018670@sys-90572:~$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
leearmy/cudnn6      ppc64le             de632816acda        20 hours ago        6.58 GB
ppc64le/ubuntu      latest              44c2eaa37868        3 days ago          150 MB
u0018670@sys-90572:~$





2017년 12월 10일 일요일

IBM Minsky (S822LC for HPC) 에서 nvidia-docker 설치 - Ubuntu

ppc64le 환경에서 nvidia-docker 를 설치할때마다 nvidia-docker 가 update 되고 방법이 틀려졌음.
http://hwengineer.blogspot.kr/2017/02/minsky-ubuntu-1604-xenial-ppc64le.html
https://hwengineer.blogspot.kr/2017/10/ppc64le-nvidia-docker-201710.html


2017년 12월 10일 현재 nvidia-docker 를 설치하기 위해서는 다음 사이트를 참고하였음.
https://developer.ibm.com/linuxonpower/docker-on-power/
https://nvidia.github.io/nvidia-docker/


nvidia-docker 가 설치될 환경은
-.ubuntu 16.04
-.CUDA 8.0 + cudnn 6.0.21
먼저 docker engine 설치
test_svr@ubuntu:~$
test_svr@ubuntu:~$ sudo vi /etc/apt/sources.list.d/xenial-docker.list
deb http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el/ xenial main
test_svr@ubuntu:~$ sudo apt-get update
test_svr@ubuntu:~$ sudo apt-get install docker-ce

test_svr@ubuntu:~$ dpkg -l |grep -i docker
ii  docker                                  1.5-1                                      ppc64el      System tray for KDE3/GNOME2 docklet applications
ii  docker-ce                               17.06.0~ce-0~ubuntu                        ppc64el      Docker: the open-source application container engine



docker engine 이 설치되었으면 nvidia-docker 설치
test_svr@ubuntu:~$ curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
> sudo apt-key add -
OK
test_svr@ubuntu:~$ curl -s -L https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el/nvidia-docker.list | \
>   sudo tee /etc/apt/sources.list.d/nvidia-docker.list
deb https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el /
test_svr@ubuntu:~$ sudo apt-get update
Get:1 file:/var/cuda-repo-8-0-local-ga2v2 InRelease
Ign:1 file:/var/cuda-repo-8-0-local-ga2v2 InRelease
Get:2 file:/var/nvidia-driver-local-repo-tr6 InRelease
Ign:2 file:/var/nvidia-driver-local-repo-tr6 InRelease
Get:3 file:/var/cuda-repo-8-0-local-ga2v2 Release [574 B]
Get:4 file:/var/nvidia-driver-local-repo-tr6 Release [574 B]
Get:3 file:/var/cuda-repo-8-0-local-ga2v2 Release [574 B]
Get:4 file:/var/nvidia-driver-local-repo-tr6 Release [574 B]
Get:7 http://ports.ubuntu.com/ubuntu-ports xenial-security InRelease [102 kB]
Hit:8 http://kr.ports.ubuntu.com/ubuntu-ports xenial InRelease
Get:9 https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el InRelease [1,098 B]
Get:10 http://kr.ports.ubuntu.com/ubuntu-ports xenial-updates InRelease [102 kB]
Get:11 https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el Packages [704 B]
Ign:12 http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial InRelease
Get:13 http://kr.ports.ubuntu.com/ubuntu-ports xenial-backports InRelease [102 kB]
Hit:14 http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial Release
Ign:15 http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial Release.gpg
Fetched 308 kB in 2s (111 kB/s)
Reading package lists... Done
W: The repository 'http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial Release' is not signed.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.

test_svr@ubuntu:~$ sudo apt-get install -y nvidia-docker
 Reading package lists... Done
 Building dependency tree
 Reading state information... Done
 The following NEW packages will be installed:
 nvidia-docker
 0 upgraded, 1 newly installed, 0 to remove and 84 not upgraded.
 Need to get 2,267 kB of archives.
 After this operation, 16.0 MB of additional disk space will be used.
 Get:1 https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el nvidia-docker 1.0.1-1 [2,267 kB]
 Fetched 2,267 kB in 2s (964 kB/s)
 Selecting previously unselected package nvidia-docker.
 (Reading database ... 122052 files and directories currently installed.)
 Preparing to unpack .../nvidia-docker_1.0.1-1_ppc64el.deb ...
 Unpacking nvidia-docker (1.0.1-1) ...
 Processing triggers for ureadahead (0.100.0-19) ...
 ureadahead will be reprofiled on next reboot
 Setting up nvidia-docker (1.0.1-1) ...
 Configuring user
 Setting up permissions
 Processing triggers for ureadahead (0.100.0-19) ...
test_svr@ubuntu:~$

2017년 12월 7일 목요일

내가 자주 쓰는, 기억해야 하는 파이썬 command - Ubuntu

framework 버전 확인
anaconda 를 통해 pip install 로 설치된 tensorflow 1.3.1 로 동작 확인
u0018670@sys-90257:~$ env | grep PATH
PATH=/home/u0018670/bin:/home/u0018670/.local/bin:/home/u0018670/anaconda2/bin:/opt/ibm/xlC/current/bin:/opt/ibm/xlf/current/bin:/opt/at10.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
u0018670@sys-90257:~$ python
Python 2.7.13 |Anaconda, Inc.| (default, Sep 15 2017, 20:54:27)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import tensorflow as tf
>>> print(tf.__version__)
1.3.1
>>>


powerAI toolkit 에 포함된 tensorflow 로 동작

u0018670@sys-90257:~$ . /opt/DL/tensorflow/bin/tensorflow-activate
u0018670@sys-90257:~$ env | grep PATH
LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64:/usr/local/cuda-8.0/extras/CUPTI/lib64:/opt/DL/tensorflow/lib
PATH=/home/u0018670/bin:/home/u0018670/.local/bin:/home/u0018670/anaconda2/bin:/opt/ibm/xlC/current/bin:/opt/ibm/xlf/current/bin:/opt/at10.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/opt/DL/tensorflow/bin
PYTHONPATH=/opt/DL/tensorflow/lib/python2.7/site-packages
u0018670@sys-90257:~$ python
Python 2.7.13 |Anaconda, Inc.| (default, Sep 15 2017, 20:54:27)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import tensorflow as tf
>>> print(tf.__version__)
1.1.0
>>>


python system path 확인

u0018670@sys-90257:~$ python3
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/lib/python35.zip', '/usr/lib/python3.5', '/usr/lib/python3.5/plat-powerpc64le-linux-gnu', '/usr/lib/python3.5/lib-dynload', '/home/u0018670/.local/lib/python3.5/site-packages', '/usr/local/lib/python3.5/dist-packages', '/usr/lib/python3/dist-packages']
>>>


native python3 설치 후 link 생성

native python3 를 다음과 같이 설치 후,
u0018670@sys-90257:~$ sudo apt-get install python3

jupyter 도 pip install 설치
u0018670@sys-90254:~$ pip list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
bleach (2.1.1)
decorator (4.1.2)
entrypoints (0.2.3)
html5lib (1.0b10)
ipykernel (4.6.1)
ipython (6.2.1)
ipython-genutils (0.2.0)
ipywidgets (7.0.5)
jedi (0.11.0)
Jinja2 (2.10)
jsonschema (2.6.0)
jupyter (1.0.0)
jupyter-client (5.1.0)
jupyter-console (5.2.0)
jupyter-core (4.4.0)
MarkupSafe (1.0)


그런데 jupyter 가 실행이 안됨.

u0018670@sys-90257:~$ jupyter
-bash: /home/u0018670/.local/bin/jupyter: /usr/bin/python: bad interpreter: No such file or directory

안되는 이유는 /usr/bin/python 파일이 없어서 그래서 link 시켜주면 잘 됨.
u0018670@sys-90257:~$ sudo ln -s /usr/bin/python3 /usr/bin/python
u0018670@sys-90257:~$ python
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

u0018670@sys-90257:~$ jupyter
usage: jupyter [-h] [--version] [--config-dir] [--data-dir] [--runtime-dir]
               [--paths] [--json]
               [subcommand]


theano 기본 테스트
root@3498b2e65eb1:/tmp# python
Python 3.5.4 |Anaconda, Inc.| (default, Sep 19 2017, 14:42:07)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import theano
>>> import theano.tensor as T
>>> import numpy as np
>>> x = T.dscalar('x')
>>> y = x ** 2
>>> f = theano.function(inputs=[x], outputs=y)
>>> f(1)
array(1.0)
>>> f(2)
array(4.0)
>>> f(3)
array(9.0)
>>>

jupyter notebook 에서는 다음과 같이 출력 확인

2017년 12월 6일 수요일

내가 자주쓰는, 기억해야 하는 command - Ubuntu

환경변수 확인
u0018670@sys-90254:~$ more /etc/environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
u0018670@sys-90254:~$ env | grep -i path
PATH=/home/u0018670/bin:/home/u0018670/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games

설치된 패키지 경로 확인
u0018670@sys-90254:~$ dpkg -S python-pip-whl
python-pip-whl: /usr/share/doc/python-pip-whl/changelog.Debian.gz
python-pip-whl: /usr/share/doc/python-pip-whl/copyright
python-pip-whl: /usr/share/doc/python-pip-whl
u0018670@sys-90254:~$ dpkg -S ifconfig
net-tools: /sbin/ifconfig
net-tools: /usr/share/man/de/man8/ifconfig.8.gz
linux-headers-4.4.0-103: /usr/src/linux-headers-4.4.0-103/tools/hv/hv_set_ifconfig.sh
net-tools: /usr/share/man/man8/ifconfig.8.gz
net-tools: /usr/share/man/pt_BR/man8/ifconfig.8.gz
linux-headers-4.4.0-104: /usr/src/linux-headers-4.4.0-104/tools/hv/hv_set_ifconfig.sh
net-tools: /usr/share/man/fr/man8/ifconfig.8.gz


/opt/at10.0 경로
IBM 패키지 중 advance-toolchain, xlc 등이 설치되는 경로 여기에 pip3 도 있고 gcc 도 있고... 과감히 삭제
u0018670@sys-90257:/opt/at10.0/bin$ dpkg -l |grep advance
ii  advance-toolchain-at10.0-devel      10.0-2                                     ppc64el      Advance Toolchain
ii  advance-toolchain-at10.0-mcore-libs 10.0-2                                     ppc64el      Advance Toolchain
ii  advance-toolchain-at10.0-perf       10.0-2                                     ppc64el      Advance Toolchain
ii  advance-toolchain-at10.0-runtime    10.0-2                                     ppc64el      Advance Toolchain

2017년 12월 5일 화요일

Python 2.7 과 Python 3.5 동시 사용 - Ubuntu

Anaconda2 로 python 2.7 이 설치된 환경에서 root 에 영향이 없이, 다시말해 Anaconda3 를 설치하지 않아도 python 3.5 를 사용하기 위해 다음과 같이 수행

현재 설치된 anaconda2 확인
u0018670@sys-90257:~$ which python
/home/u0018670/anaconda2/bin/python

jupyter kernal 확인
u0018670@sys-90257:~$ jupyter kernelspec list
Available kernels:
  python2    /home/u0018670/anaconda2/share/jupyter/kernels/python2

jupyter notebook 을 실행해봐도 Python2 만 확인

u0018670@sys-90257:~$ conda update conda
Fetching package metadata ...........
Solving package specifications: .

# All requested packages already installed.
# packages in environment at /home/u0018670/anaconda2:
#
conda                     4.3.27                   py27_0

python 3.5 환경으로 업데이트
u0018670@sys-90257:~$ conda create -n py35 python=3.5 anaconda
Fetching package metadata ...........
Solving package specifications: .

Package plan for installation in environment /home/u0018670/anaconda2/envs/py35:

The following NEW packages will be INSTALLED:

    alabaster:                0.7.10-py35_0

    anaconda:                 5.0.0-py35h419c926_0
    anaconda-client:          1.6.3-py35_0
    anaconda-project:         0.6.0-py35_0
    asn1crypto:               0.22.0-py35_0
    astroid:                  1.5.3-py35_0
    astropy:                  2.0.1-np113py35_1
    babel:                    2.5.0-py35_0
    backports:                1.0-py35_0
    beautifulsoup4:           4.6.0-py35_0
    bitarray:                 0.8.1-py35_0
......................
Proceed ([y]/n)? y

python-3.5.4-0 100% |#####################################################################################| Time: 0:00:01  14.46 MB/s
alabaster-0.7. 100% |#####################################################################################| Time: 0:00:00  15.39 MB/s
asn1crypto-0.2 100% |#####################################################################################| Time: 0:00:00  16.71 MB/s
backports-1.0- 100% |#####################################################################################| Time: 0:00:00   3.98 MB/s
beautifulsoup4 100% |#####################################################################################| Time: 0:00:00  18.22 MB/s
bitarray-0.8.1 100% |#####################################################################################| Time: 0:00:00  25.68 MB/s
boto-2.48.0-py 100% |#####################################################################################| Time: 0:00:00   3.83 MB/s
......................
anaconda-5.0.0 100% |#####################################################################################| Time: 0:00:00  12.86 MB/s
#
# To activate this environment, use:
# > source activate py35
#
# To deactivate an active environment, use:
# > source deactivate
#
u0018670@sys-90257:~$ source activate py35
(py35) u0018670@sys-90257:~$ jupyter notebook

(py35) u0018670@sys-90257:~/anaconda2$ which python
/home/u0018670/anaconda2/envs/py35/bin/python

(py35) u0018670@sys-90257:~/anaconda2$ source deactivate
u0018670@sys-90257:~/anaconda2$ 


추가적으로, 특정 버전의 python 패키지를 설치할때는 다음과 같이
그냥 keras 설치를 해버리면 최신 버전이 설치됨.
nimbix@JARVICENAE-0A0A183D:~$ pip install keras
Collecting keras
  Downloading Keras-2.1.2-py2.py3-none-any.whl (304kB)
    100% |████████████████████████████████| 307kB 4.3MB/s
Requirement already satisfied: scipy>=0.14 in ./anaconda3/lib/python3.6/site-packages (from keras)
Requirement already satisfied: six>=1.9.0 in ./anaconda3/lib/python3.6/site-packages (from keras)
Requirement already satisfied: pyyaml in ./anaconda3/lib/python3.6/site-packages (from keras)
Requirement already satisfied: numpy>=1.9.1 in ./anaconda3/lib/python3.6/site-packages (from keras)
Installing collected packages: keras
Successfully installed keras-2.1.2
 

그런데, 
nimbix@JARVICENAE-0A0A183D:~$ pip install keras==1.2
Collecting keras==1.2
  Downloading Keras-1.2.0.tar.gz (167kB)
    100% |████████████████████████████████| 174kB 6.4MB/s
Collecting theano (from keras==1.2)
  Downloading Theano-1.0.1.tar.gz (2.8MB)
    100% |████████████████████████████████| 2.8MB 506kB/s
Requirement already satisfied: pyyaml in ./anaconda3/lib/python3.6/site-packages (from keras==1.2)
Requirement already satisfied: six in ./anaconda3/lib/python3.6/site-packages (from keras==1.2)
Requirement already satisfied: numpy>=1.9.1 in ./anaconda3/lib/python3.6/site-packages (from theano->keras==1.2)
Requirement already satisfied: scipy>=0.14 in ./anaconda3/lib/python3.6/site-packages (from theano->keras==1.2)
Building wheels for collected packages: keras, theano
  Running setup.py bdist_wheel for keras ... done
  Stored in directory: /home/nimbix/.cache/pip/wheels/f8/2c/8e/27d952128220b1acf6fcd5e692269c6e8d60744c15fd160c2c
  Running setup.py bdist_wheel for theano ... done
  Stored in directory: /home/nimbix/.cache/pip/wheels/46/a2/7d/b4cac381d5151daa9f9e0b3e4e4b65edaea6355ae296c97cf2
Successfully built keras theano
Installing collected packages: theano, keras
  Found existing installation: Keras 2.1.2
    Uninstalling Keras-2.1.2:
      Successfully uninstalled Keras-2.1.2
Successfully installed keras-1.2.0 theano-1.0.1
 

기존 2.1.2 를 자동으로 삭제하고 1.2 가 설치됨. 

2017년 12월 4일 월요일

폐쇄망에서 python 패키지 설치 - Ubuntu

먼저, 인터넷이 되는 곳에서 원하는 패키지를 다음과 같이 다운로드 받는다.

online@ubuntu:~$ pip install --download . tensorflow
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting tensorflow
  Downloading tensorflow-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl (40.7MB)
    100% |████████████████████████████████| 40.8MB 23kB/s
  Saved ./tensorflow-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting enum34>=1.1.6 (from tensorflow)
  Downloading enum34-1.1.6-py2-none-any.whl
  Saved ./enum34-1.1.6-py2-none-any.whl
Collecting backports.weakref>=1.0rc1 (from tensorflow)
  Downloading backports.weakref-1.0.post1-py2.py3-none-any.whl
  Saved ./backports.weakref-1.0.post1-py2.py3-none-any.whl
Collecting wheel (from tensorflow)
  Downloading wheel-0.30.0-py2.py3-none-any.whl (49kB)
    100% |████████████████████████████████| 51kB 6.9MB/s
  Saved ./wheel-0.30.0-py2.py3-none-any.whl
Collecting mock>=2.0.0 (from tensorflow)
  Downloading mock-2.0.0-py2.py3-none-any.whl (56kB)
    100% |████████████████████████████████| 61kB 2.7MB/s
  Saved ./mock-2.0.0-py2.py3-none-any.whl
Collecting tensorflow-tensorboard<0.5.0,>=0.4.0rc1 (from tensorflow)
  Downloading tensorflow_tensorboard-0.4.0rc3-py2-none-any.whl (1.7MB)
    100% |████████████████████████████████| 1.7MB 537kB/s
  Saved ./tensorflow_tensorboard-0.4.0rc3-py2-none-any.whl
Collecting numpy>=1.12.1 (from tensorflow)
  Downloading numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl (16.6MB)
    100% |████████████████████████████████| 16.7MB 66kB/s
  Saved ./numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Collecting protobuf>=3.3.0 (from tensorflow)
  Downloading protobuf-3.5.0.post1-cp27-cp27mu-manylinux1_x86_64.whl (6.4MB)
    100% |████████████████████████████████| 6.4MB 166kB/s
  Saved ./protobuf-3.5.0.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting six>=1.10.0 (from tensorflow)
  Downloading six-1.11.0-py2.py3-none-any.whl
  Saved ./six-1.11.0-py2.py3-none-any.whl
Collecting funcsigs>=1; python_version < "3.3" (from mock>=2.0.0->tensorflow)
  Downloading funcsigs-1.0.2-py2.py3-none-any.whl
  Saved ./funcsigs-1.0.2-py2.py3-none-any.whl
Collecting pbr>=0.11 (from mock>=2.0.0->tensorflow)
  Downloading pbr-3.1.1-py2.py3-none-any.whl (99kB)
    100% |████████████████████████████████| 102kB 1.6MB/s
  Saved ./pbr-3.1.1-py2.py3-none-any.whl
Collecting bleach==1.5.0 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
  Downloading bleach-1.5.0-py2.py3-none-any.whl
  Saved ./bleach-1.5.0-py2.py3-none-any.whl
Collecting futures>=3.1.1; python_version < "3.2" (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
  Downloading futures-3.2.0-py2-none-any.whl
  Saved ./futures-3.2.0-py2-none-any.whl
Collecting markdown>=2.6.8 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
  Downloading Markdown-2.6.9.tar.gz (271kB)
    100% |████████████████████████████████| 276kB 1.3MB/s
  Saved ./Markdown-2.6.9.tar.gz
Collecting werkzeug>=0.11.10 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
  Downloading Werkzeug-0.12.2-py2.py3-none-any.whl (312kB)
    100% |████████████████████████████████| 317kB 1.3MB/s
  Saved ./Werkzeug-0.12.2-py2.py3-none-any.whl
Collecting html5lib==0.9999999 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
  Downloading html5lib-0.9999999.tar.gz (889kB)
    100% |████████████████████████████████| 890kB 744kB/s
  Saved ./html5lib-0.9999999.tar.gz
Collecting setuptools (from protobuf>=3.3.0->tensorflow)
  Downloading setuptools-38.2.4-py2.py3-none-any.whl (489kB)
    100% |████████████████████████████████| 491kB 1.0MB/s
  Saved ./setuptools-38.2.4-py2.py3-none-any.whl
Successfully downloaded tensorflow enum34 backports.weakref wheel mock tensorflow-tensorboard numpy protobuf six funcsigs pbr bleach futures markdown werkzeug html5lib setuptools


다운로드 받은 패키지들은 의존성에 의해 관련된 패키지들도 함께 다운받아지며 이것들을 폐쇄망 서버로 이동(tensorflow 디렉토리) 후, 다음과 같이 설치.

offline@ubuntu:~/tensorflow$ pip install --no-index --find-links=. tensorflow
Collecting tensorflow
Requirement already satisfied: enum34>=1.1.6 in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow)
Collecting backports.weakref>=1.0rc1 (from tensorflow)
Requirement already satisfied: wheel in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow)
Collecting mock>=2.0.0 (from tensorflow)
Collecting tensorflow-tensorboard<0.5.0,>=0.4.0rc1 (from tensorflow)
Requirement already satisfied: numpy>=1.12.1 in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow)
Collecting protobuf>=3.3.0 (from tensorflow)
Requirement already satisfied: six>=1.10.0 in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/testuser01/anaconda2/lib/python2.7/site-packages (from mock>=2.0.0->tensorflow)
Collecting pbr>=0.11 (from mock>=2.0.0->tensorflow)
Collecting bleach==1.5.0 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Requirement already satisfied: futures>=3.1.1; python_version < "3.2" in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Collecting markdown>=2.6.8 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Requirement already satisfied: werkzeug>=0.11.10 in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Collecting html5lib==0.9999999 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Requirement already satisfied: setuptools in /home/testuser01/anaconda2/lib/python2.7/site-packages (from protobuf>=3.3.0->tensorflow)
Building wheels for collected packages: markdown, html5lib
  Running setup.py bdist_wheel for markdown ... done
  Stored in directory: /home/testuser01/.cache/pip/wheels/51/36/90/47aaba211d675deb9af500b7a894541103b283c5f18e0ef529
  Running setup.py bdist_wheel for html5lib ... done
  Stored in directory: /home/testuser01/.cache/pip/wheels/22/7a/d3/e96250b678c5b3af1d50336781a06e05568d81251c10440b4a
Successfully built markdown html5lib
Installing collected packages: backports.weakref, pbr, mock, html5lib, bleach, protobuf, markdown, tensorflow-tensorboard, tensorflow
  Found existing installation: html5lib 0.999999999
    Uninstalling html5lib-0.999999999:
      Successfully uninstalled html5lib-0.999999999
  Found existing installation: bleach 2.0.0
    Uninstalling bleach-2.0.0:
      Successfully uninstalled bleach-2.0.0
Successfully installed backports.weakref-1.0.post1 bleach-1.5.0 html5lib-0.9999999 markdown-2.6.9 mock-2.0.0 pbr-3.1.1 protobuf-3.5.0.post1 tensorflow-1.4.0 tensorflow-tensorboard-0.4.0rc3
offline@ubuntu:~/tensorflow$ conda list | grep tensorflow
tensorflow                1.4.0                     <pip>
tensorflow-tensorboard    0.4.0rc3                  <pip>