설치된 docker engine 확인 u0018670@sys-90572:~$ dpkg -l |grep docker ii docker.io 1.13.1-0ubuntu1~16.04.2 ppc64el Linux container runtime ii runc 1.0.0~rc2+docker1.13.1-0ubuntu1~16.04.1 ppc64el Open Container Project - runtime 아무런 docker image 가 없는 것을 확인 u0018670@sys-90572:~$ sudo docker images REPOSITORY TAG IMAGE ID CREATED SIZE 그래서, ppc64le/ubuntu docker image 를 실행시키면 latest 로 docker hub 로 부터 docker tag 가 latest 인 것을 download 후 docker 실행 u0018670@sys-90572:~$ sudo docker run -it ppc64le/ubuntu Unable to find image 'ppc64le/ubuntu:latest' locally latest: Pulling from ppc64le/ubuntu 586a07d977d9: Pull complete 2bf911559ec0: Pull complete 48999c060756: Pull complete fa5e5494889f: Pull complete 3bd4f80d03d3: Pull complete Digest: sha256:79d11934f87adb78614d5ca93092f6681ca154c2bd9df56a7027615c56d53e41 Status: Downloaded newer image for ppc64le/ubuntu:latest root@b916c38bbbb7:/# ls <== docker container 안 bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var root@b916c38bbbb7:/# exit exit 생성된 docker image 확인 u0018670@sys-90572:~$ sudo docker images REPOSITORY TAG IMAGE ID CREATED SIZE ppc64le/ubuntu latest 44c2eaa37868 3 days ago 150 MB 수행중인 container 확인 u0018670@sys-90572:~$ sudo docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES b916c38bbbb7 ppc64le/ubuntu "/bin/bash" 54 seconds ago Exited (0) 17 seconds ago epic_cori
수행중인 container 를 image 로 commit u0018670@sys-90727:~$ sudo docker commit 58d0d814ae31 ppc64le/ubuntu:latest sha256:4de2af599de3a7a0aa2351a9fcb6e9b8cbaa8f3a345d6fed03291477097da82d u0018670@sys-90727:~$ sudo docker images REPOSITORY TAG IMAGE ID CREATED SIZE ppc64le/ubuntu latest 4de2af599de3 About a minute ago 260MB ppc64le/ubuntu 16.04 44c2eaa37868 3 weeks ago 150MB 수행중인 container 삭제 u0018670@sys-90572:~$ sudo docker rm epic_cori epic_cori u0018670@sys-90572:~$ sudo docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES u0018670@sys-90572:~$ 한꺼번에 모든 container 삭제는 docker rm $(docker ps -a -q) 다음은 2개의 container 가 시작된 상황. (ppc64le/ubuntu 이미지로 eager_hugle, quirky_pike) u0018670@sys-90572:~$ sudo docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES eaf515d4a2c6 ppc64le/ubuntu "/bin/bash" 16 seconds ago Exited (0) 12 seconds ago eager_hugle 4e3e7338019b ppc64le/ubuntu "/bin/bash" About a minute ago Exited (0) 21 seconds ago quirky_pike container 수행 중이기에 삭제 되지 않는 docker image, rm 옵션으로 container 를 먼저 삭제 후 image 삭제 u0018670@sys-90572:~$ sudo docker rmi ppc64le/ubuntu Error response from daemon: conflict: unable to remove repository reference "ppc64le/ubuntu" (must force) - container 4e3e7338019b is using its referenced image 44c2eaa37868 u0018670@sys-90572:~$ docker hub 에 생성된 이미지 upload u0018670@sys-90461:~/data$ sudo docker images REPOSITORY TAG IMAGE ID CREATED SIZE leearmy/cudnn6 ppc64le de632816acda About an hour ago 6.58GB leearmy/ubuntu 16.04 23296ffbcbfc 4 hours ago 359MB ppc64le/ubuntu 16.04 44c2eaa37868 3 days ago 150MB 다음 명령어를 수행하기 위해서는 먼저 https://hub.docker.com/ 에 계정과 repository 를 생성하여야 함 u0018670@sys-90461:~/data$ sudo docker login --username=leearmy Password: Login Succeeded u0018670@sys-90461:~/data$ sudo docker push leearmy/cudnn6:ppc64le The push refers to a repository [docker.io/leearmy/cudnn6] 026ec1f6d915: Pushed c963a10dda0d: Pushed 4a08cdc31ed0: Pushed 49076433d00c: Pushed 8788fee75149: Pushed d9561c8b4bf3: Pushed 54ac221998f0: Pushed b10475677e3b: Pushed adf3a2526761: Pushed e8449de7ace1: Pushed 0c5b7df1df7c: Pushed b1dc141f6073: Mounted from ppc64le/ubuntu 448bb1e132fc: Mounted from ppc64le/ubuntu e7f4651b678d: Mounted from ppc64le/ubuntu ee8f1c150908: Mounted from ppc64le/ubuntu 258cc4d5502c: Mounted from ppc64le/ubuntu ppc64le: digest: sha256:3df7ee55f48c08fe7dc836c60053867dac28ada8b7eb9b9413b09e5361e4937c size: 3690 다음과 같이 https://hub.docker.com/ 에서 올려진 이미지 확인
docker hub 에 올려진 이미지는 다른 서버에서 pull u0018670@sys-90572:~$ sudo docker images REPOSITORY TAG IMAGE ID CREATED SIZE ppc64le/ubuntu latest 44c2eaa37868 3 days ago 150 MB u0018670@sys-90572:~$ sudo docker pull leearmy/cudnn6:ppc64le ppc64le: Pulling from leearmy/cudnn6 586a07d977d9: Already exists 2bf911559ec0: Already exists 48999c060756: Already exists fa5e5494889f: Already exists 3bd4f80d03d3: Already exists 3d482c0c4d09: Pull complete 9e891e52327a: Pull complete a40daf5d9d7f: Pull complete 2ca681ce2118: Pull complete 111379f9235b: Pull complete 9db8b0fb9301: Pull complete b2b39482e9b8: Pull complete 93f5e3ec7d59: Pull complete 38e0ece0a1d4: Pull complete dbc7427ab4ea: Pull complete f1cc2b939ac9: Pull complete Digest: sha256:3df7ee55f48c08fe7dc836c60053867dac28ada8b7eb9b9413b09e5361e4937c Status: Downloaded newer image for leearmy/cudnn6:ppc64le u0018670@sys-90572:~$ sudo docker images REPOSITORY TAG IMAGE ID CREATED SIZE leearmy/cudnn6 ppc64le de632816acda 20 hours ago 6.58 GB ppc64le/ubuntu latest 44c2eaa37868 3 days ago 150 MBu0018670@sys-90572:~$
nvidia-docker 가 설치될 환경은 -.ubuntu 16.04 -.CUDA 8.0 + cudnn 6.0.21 먼저 docker engine 설치 test_svr@ubuntu:~$ test_svr@ubuntu:~$ sudo vi /etc/apt/sources.list.d/xenial-docker.list deb http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el/ xenial main test_svr@ubuntu:~$ sudo apt-get update test_svr@ubuntu:~$ sudo apt-get install docker-ce test_svr@ubuntu:~$ dpkg -l |grep -i docker ii docker 1.5-1 ppc64el System tray for KDE3/GNOME2 docklet applications ii docker-ce 17.06.0~ce-0~ubuntu ppc64el Docker: the open-source application container engine
docker engine 이 설치되었으면 nvidia-docker 설치 test_svr@ubuntu:~$curl -s-L https://nvidia.github.io/nvidia-docker/gpgkey | \ > sudo apt-key add - OK test_svr@ubuntu:~$ curl -s-L https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el/nvidia-docker.list | \ > sudo tee /etc/apt/sources.list.d/nvidia-docker.list deb https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el / test_svr@ubuntu:~$ sudo apt-get update Get:1 file:/var/cuda-repo-8-0-local-ga2v2 InRelease Ign:1 file:/var/cuda-repo-8-0-local-ga2v2 InRelease Get:2 file:/var/nvidia-driver-local-repo-tr6 InRelease Ign:2 file:/var/nvidia-driver-local-repo-tr6 InRelease Get:3 file:/var/cuda-repo-8-0-local-ga2v2 Release [574 B] Get:4 file:/var/nvidia-driver-local-repo-tr6 Release [574 B] Get:3 file:/var/cuda-repo-8-0-local-ga2v2 Release [574 B] Get:4 file:/var/nvidia-driver-local-repo-tr6 Release [574 B] Get:7 http://ports.ubuntu.com/ubuntu-ports xenial-security InRelease [102 kB] Hit:8 http://kr.ports.ubuntu.com/ubuntu-ports xenial InRelease Get:9 https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el InRelease [1,098 B] Get:10 http://kr.ports.ubuntu.com/ubuntu-ports xenial-updates InRelease [102 kB] Get:11 https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el Packages [704 B] Ign:12 http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial InRelease Get:13 http://kr.ports.ubuntu.com/ubuntu-ports xenial-backports InRelease [102 kB] Hit:14 http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial Release Ign:15 http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial Release.gpg Fetched 308 kB in 2s (111 kB/s) Reading package lists... Done W: The repository 'http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial Release' is not signed. N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use. N: See apt-secure(8) manpage for repository creation and user configuration details. test_svr@ubuntu:~$ sudo apt-get install -y nvidia-docker Reading package lists... Done Building dependency tree Reading state information... Done The following NEW packages will be installed: nvidia-docker 0 upgraded, 1 newly installed, 0 to remove and 84 not upgraded. Need to get 2,267 kB of archives. After this operation, 16.0 MB of additional disk space will be used. Get:1 https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el nvidia-docker 1.0.1-1 [2,267 kB] Fetched 2,267 kB in 2s (964 kB/s) Selecting previously unselected package nvidia-docker. (Reading database ... 122052 files and directories currently installed.) Preparing to unpack .../nvidia-docker_1.0.1-1_ppc64el.deb ... Unpacking nvidia-docker (1.0.1-1) ... Processing triggers for ureadahead (0.100.0-19) ... ureadahead will be reprofiled on next reboot Setting up nvidia-docker (1.0.1-1) ... Configuring user Setting up permissions Processing triggers for ureadahead (0.100.0-19) ... test_svr@ubuntu:~$
framework 버전 확인 anaconda 를 통해 pip install 로 설치된 tensorflow 1.3.1 로 동작 확인 u0018670@sys-90257:~$ env | grep PATH PATH=/home/u0018670/bin:/home/u0018670/.local/bin:/home/u0018670/anaconda2/bin:/opt/ibm/xlC/current/bin:/opt/ibm/xlf/current/bin:/opt/at10.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games u0018670@sys-90257:~$ python Python 2.7.13 |Anaconda, Inc.| (default, Sep 15 2017, 20:54:27) [GCC 4.8.4] on linux2 Type "help", "copyright", "credits" or "license" for more information. Anaconda is brought to you by Continuum Analytics. Please check out: http://continuum.io/thanks and https://anaconda.org >>> import tensorflow as tf >>> print(tf.__version__) 1.3.1 >>> powerAI toolkit 에 포함된 tensorflow 로 동작 u0018670@sys-90257:~$ . /opt/DL/tensorflow/bin/tensorflow-activate u0018670@sys-90257:~$ env | grep PATH LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64:/usr/local/cuda-8.0/extras/CUPTI/lib64:/opt/DL/tensorflow/lib PATH=/home/u0018670/bin:/home/u0018670/.local/bin:/home/u0018670/anaconda2/bin:/opt/ibm/xlC/current/bin:/opt/ibm/xlf/current/bin:/opt/at10.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/opt/DL/tensorflow/bin PYTHONPATH=/opt/DL/tensorflow/lib/python2.7/site-packages u0018670@sys-90257:~$ python Python 2.7.13 |Anaconda, Inc.| (default, Sep 15 2017, 20:54:27) [GCC 4.8.4] on linux2 Type "help", "copyright", "credits" or "license" for more information. Anaconda is brought to you by Continuum Analytics. Please check out: http://continuum.io/thanks and https://anaconda.org >>> import tensorflow as tf >>> print(tf.__version__) 1.1.0 >>> python system path 확인 u0018670@sys-90257:~$ python3 Python 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path ['', '/usr/lib/python35.zip', '/usr/lib/python3.5', '/usr/lib/python3.5/plat-powerpc64le-linux-gnu', '/usr/lib/python3.5/lib-dynload', '/home/u0018670/.local/lib/python3.5/site-packages', '/usr/local/lib/python3.5/dist-packages', '/usr/lib/python3/dist-packages'] >>> native python3 설치 후 link 생성 native python3 를 다음과 같이 설치 후, u0018670@sys-90257:~$ sudo apt-get install python3 jupyter 도 pip install 설치 u0018670@sys-90254:~$ pip list DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning. bleach (2.1.1) decorator (4.1.2) entrypoints (0.2.3) html5lib (1.0b10) ipykernel (4.6.1) ipython (6.2.1) ipython-genutils (0.2.0) ipywidgets (7.0.5) jedi (0.11.0) Jinja2 (2.10) jsonschema (2.6.0) jupyter (1.0.0) jupyter-client (5.1.0) jupyter-console (5.2.0) jupyter-core (4.4.0) MarkupSafe (1.0) 그런데 jupyter 가 실행이 안됨. u0018670@sys-90257:~$ jupyter -bash: /home/u0018670/.local/bin/jupyter: /usr/bin/python: bad interpreter: No such file or directory 안되는 이유는 /usr/bin/python 파일이 없어서 그래서 link 시켜주면 잘 됨. u0018670@sys-90257:~$ sudo ln -s /usr/bin/python3 /usr/bin/python u0018670@sys-90257:~$ python Python 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> u0018670@sys-90257:~$ jupyter usage: jupyter [-h] [--version] [--config-dir] [--data-dir] [--runtime-dir] [--paths] [--json] [subcommand] theano 기본 테스트 root@3498b2e65eb1:/tmp# python Python 3.5.4 |Anaconda, Inc.| (default, Sep 19 2017, 14:42:07) [GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import theano >>> import theano.tensor as T >>> import numpy as np >>> x = T.dscalar('x') >>> y = x ** 2 >>> f = theano.function(inputs=[x], outputs=y) >>> f(1) array(1.0) >>> f(2) array(4.0) >>> f(3) array(9.0) >>> jupyter notebook 에서는 다음과 같이 출력 확인
/opt/at10.0 경로 IBM 패키지 중 advance-toolchain, xlc 등이 설치되는 경로 여기에 pip3 도 있고 gcc 도 있고... 과감히 삭제 u0018670@sys-90257:/opt/at10.0/bin$ dpkg -l |grep advance ii advance-toolchain-at10.0-devel 10.0-2 ppc64el Advance Toolchain ii advance-toolchain-at10.0-mcore-libs 10.0-2 ppc64el Advance Toolchain ii advance-toolchain-at10.0-perf 10.0-2 ppc64el Advance Toolchain ii advance-toolchain-at10.0-runtime 10.0-2 ppc64el Advance Toolchain
Anaconda2 로 python 2.7 이 설치된 환경에서 root 에 영향이 없이, 다시말해 Anaconda3 를 설치하지 않아도 python 3.5 를 사용하기 위해 다음과 같이 수행 현재 설치된 anaconda2 확인 u0018670@sys-90257:~$ which python /home/u0018670/anaconda2/bin/python jupyter kernal 확인 u0018670@sys-90257:~$ jupyter kernelspec list Available kernels: python2 /home/u0018670/anaconda2/share/jupyter/kernels/python2 jupyter notebook 을 실행해봐도 Python2 만 확인
u0018670@sys-90257:~$ conda update conda Fetching package metadata ........... Solving package specifications: . # All requested packages already installed. # packages in environment at /home/u0018670/anaconda2: # conda 4.3.27 py27_0 python 3.5 환경으로 업데이트 u0018670@sys-90257:~$ conda create -n py35 python=3.5 anaconda Fetching package metadata ........... Solving package specifications: . Package plan for installation in environment /home/u0018670/anaconda2/envs/py35: The following NEW packages will be INSTALLED: alabaster: 0.7.10-py35_0 anaconda: 5.0.0-py35h419c926_0 anaconda-client: 1.6.3-py35_0 anaconda-project: 0.6.0-py35_0 asn1crypto: 0.22.0-py35_0 astroid: 1.5.3-py35_0 astropy: 2.0.1-np113py35_1 babel: 2.5.0-py35_0 backports: 1.0-py35_0 beautifulsoup4: 4.6.0-py35_0 bitarray: 0.8.1-py35_0 ...................... Proceed ([y]/n)? y python-3.5.4-0 100% |#####################################################################################| Time: 0:00:01 14.46 MB/s alabaster-0.7. 100% |#####################################################################################| Time: 0:00:00 15.39 MB/s asn1crypto-0.2 100% |#####################################################################################| Time: 0:00:00 16.71 MB/s backports-1.0- 100% |#####################################################################################| Time: 0:00:00 3.98 MB/s beautifulsoup4 100% |#####################################################################################| Time: 0:00:00 18.22 MB/s bitarray-0.8.1 100% |#####################################################################################| Time: 0:00:00 25.68 MB/s boto-2.48.0-py 100% |#####################################################################################| Time: 0:00:00 3.83 MB/s ...................... anaconda-5.0.0 100% |#####################################################################################| Time: 0:00:00 12.86 MB/s # # To activate this environment, use: # > source activate py35 # # To deactivate an active environment, use: # > source deactivate # u0018670@sys-90257:~$ source activate py35 (py35) u0018670@sys-90257:~$ jupyter notebook
(py35) u0018670@sys-90257:~/anaconda2$ which python /home/u0018670/anaconda2/envs/py35/bin/python (py35) u0018670@sys-90257:~/anaconda2$ source deactivate u0018670@sys-90257:~/anaconda2$
추가적으로, 특정 버전의 python 패키지를 설치할때는 다음과 같이 그냥 keras 설치를 해버리면 최신 버전이 설치됨. nimbix@JARVICENAE-0A0A183D:~$ pip install keras Collecting keras Downloading Keras-2.1.2-py2.py3-none-any.whl (304kB) 100% |████████████████████████████████| 307kB 4.3MB/s Requirement already satisfied: scipy>=0.14 in ./anaconda3/lib/python3.6/site-packages (from keras) Requirement already satisfied: six>=1.9.0 in ./anaconda3/lib/python3.6/site-packages (from keras) Requirement already satisfied: pyyaml in ./anaconda3/lib/python3.6/site-packages (from keras) Requirement already satisfied: numpy>=1.9.1 in ./anaconda3/lib/python3.6/site-packages (from keras) Installing collected packages: keras Successfully installed keras-2.1.2
그런데, nimbix@JARVICENAE-0A0A183D:~$ pip install keras==1.2 Collecting keras==1.2 Downloading Keras-1.2.0.tar.gz (167kB) 100% |████████████████████████████████| 174kB 6.4MB/s Collecting theano (from keras==1.2) Downloading Theano-1.0.1.tar.gz (2.8MB) 100% |████████████████████████████████| 2.8MB 506kB/s Requirement already satisfied: pyyaml in ./anaconda3/lib/python3.6/site-packages (from keras==1.2) Requirement already satisfied: six in ./anaconda3/lib/python3.6/site-packages (from keras==1.2) Requirement already satisfied: numpy>=1.9.1 in ./anaconda3/lib/python3.6/site-packages (from theano->keras==1.2) Requirement already satisfied: scipy>=0.14 in ./anaconda3/lib/python3.6/site-packages (from theano->keras==1.2) Building wheels for collected packages: keras, theano Running setup.py bdist_wheel for keras ... done Stored in directory: /home/nimbix/.cache/pip/wheels/f8/2c/8e/27d952128220b1acf6fcd5e692269c6e8d60744c15fd160c2c Running setup.py bdist_wheel for theano ... done Stored in directory: /home/nimbix/.cache/pip/wheels/46/a2/7d/b4cac381d5151daa9f9e0b3e4e4b65edaea6355ae296c97cf2 Successfully built keras theano Installing collected packages: theano, keras Found existing installation: Keras 2.1.2 Uninstalling Keras-2.1.2: Successfully uninstalled Keras-2.1.2 Successfully installed keras-1.2.0 theano-1.0.1
