이 블로그 검색

2017년 12월 18일 월요일

docker 기본 사용 - Ubuntu

설치된 docker engine 확인
u0018670@sys-90572:~$ dpkg -l |grep docker
ii  docker.io                           1.13.1-0ubuntu1~16.04.2                    ppc64el      Linux container runtime
ii  runc                                1.0.0~rc2+docker1.13.1-0ubuntu1~16.04.1    ppc64el      Open Container Project - runtime


아무런 docker image 가 없는 것을 확인
u0018670@sys-90572:~$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE


그래서, ppc64le/ubuntu docker image 를 실행시키면 latest 로 docker hub 로 부터 docker tag 가 latest 인 것을 download 후 docker 실행
u0018670@sys-90572:~$ sudo docker run -it ppc64le/ubuntu
Unable to find image 'ppc64le/ubuntu:latest' locally
latest: Pulling from ppc64le/ubuntu
586a07d977d9: Pull complete
2bf911559ec0: Pull complete
48999c060756: Pull complete
fa5e5494889f: Pull complete
3bd4f80d03d3: Pull complete
Digest: sha256:79d11934f87adb78614d5ca93092f6681ca154c2bd9df56a7027615c56d53e41
Status: Downloaded newer image for ppc64le/ubuntu:latest

root@b916c38bbbb7:/# ls     <== docker container 안
bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
root@b916c38bbbb7:/# exit

exit

생성된 docker image 확인
u0018670@sys-90572:~$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
ppc64le/ubuntu      latest              44c2eaa37868        3 days ago          150 MB


수행중인 container 확인
u0018670@sys-90572:~$ sudo docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                      PORTS               NAMES
b916c38bbbb7        ppc64le/ubuntu      "/bin/bash"         54 seconds ago      Exited (0) 17 seconds ago                       epic_cori


exit 한 container 재시작
u0018670@sys-90727:~$ sudo docker start confident_euclid
confident_euclid
u0018670@sys-90727:~$ sudo docker attach confident_euclid
root@0f647ddac01a:/#


수행중인 container 를 image 로 commit
u0018670@sys-90727:~$ sudo docker commit 58d0d814ae31 ppc64le/ubuntu:latest
sha256:4de2af599de3a7a0aa2351a9fcb6e9b8cbaa8f3a345d6fed03291477097da82d
u0018670@sys-90727:~$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED              SIZE
ppc64le/ubuntu      latest              4de2af599de3        About a minute ago   260MB
ppc64le/ubuntu      16.04               44c2eaa37868        3 weeks ago          150MB


수행중인 container 삭제
u0018670@sys-90572:~$ sudo docker rm epic_cori
epic_cori

u0018670@sys-90572:~$ sudo docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

u0018670@sys-90572:~$
한꺼번에 모든 container 삭제는 docker rm $(docker ps -a -q)

다음은 2개의 container 가 시작된 상황. (ppc64le/ubuntu 이미지로 eager_hugle, quirky_pike)
u0018670@sys-90572:~$ sudo docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED              STATUS                      PORTS               NAMES
eaf515d4a2c6        ppc64le/ubuntu      "/bin/bash"         16 seconds ago       Exited (0) 12 seconds ago                       eager_hugle
4e3e7338019b        ppc64le/ubuntu      "/bin/bash"         About a minute ago   Exited (0) 21 seconds ago                       quirky_pike


container 수행 중이기에 삭제 되지 않는 docker image, rm 옵션으로 container 를 먼저 삭제 후 image 삭제
u0018670@sys-90572:~$ sudo docker rmi ppc64le/ubuntu
Error response from daemon: conflict: unable to remove repository reference "ppc64le/ubuntu" (must force) - container 4e3e7338019b is using its referenced image 44c2eaa37868
u0018670@sys-90572:~$


docker hub 에 생성된 이미지 upload
u0018670@sys-90461:~/data$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
leearmy/cudnn6      ppc64le             de632816acda        About an hour ago   6.58GB
leearmy/ubuntu      16.04               23296ffbcbfc        4 hours ago         359MB
ppc64le/ubuntu      16.04               44c2eaa37868        3 days ago          150MB

다음 명령어를 수행하기 위해서는 먼저 https://hub.docker.com/ 에 계정과 repository 를 생성하여야 함
u0018670@sys-90461:~/data$ sudo docker login --username=leearmy
Password:
Login Succeeded
u0018670@sys-90461:~/data$ sudo docker push leearmy/cudnn6:ppc64le
The push refers to a repository [docker.io/leearmy/cudnn6]
026ec1f6d915: Pushed
c963a10dda0d: Pushed
4a08cdc31ed0: Pushed
49076433d00c: Pushed
8788fee75149: Pushed
d9561c8b4bf3: Pushed
54ac221998f0: Pushed
b10475677e3b: Pushed
adf3a2526761: Pushed
e8449de7ace1: Pushed
0c5b7df1df7c: Pushed
b1dc141f6073: Mounted from ppc64le/ubuntu
448bb1e132fc: Mounted from ppc64le/ubuntu
e7f4651b678d: Mounted from ppc64le/ubuntu
ee8f1c150908: Mounted from ppc64le/ubuntu
258cc4d5502c: Mounted from ppc64le/ubuntu
ppc64le: digest: sha256:3df7ee55f48c08fe7dc836c60053867dac28ada8b7eb9b9413b09e5361e4937c size: 3690


다음과 같이 https://hub.docker.com/ 에서 올려진 이미지 확인


docker hub 에 올려진 이미지는 다른 서버에서 pull
u0018670@sys-90572:~$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
ppc64le/ubuntu      latest              44c2eaa37868        3 days ago          150 MB
u0018670@sys-90572:~$ sudo docker pull leearmy/cudnn6:ppc64le
ppc64le: Pulling from leearmy/cudnn6
586a07d977d9: Already exists
2bf911559ec0: Already exists
48999c060756: Already exists
fa5e5494889f: Already exists
3bd4f80d03d3: Already exists
3d482c0c4d09: Pull complete
9e891e52327a: Pull complete
a40daf5d9d7f: Pull complete
2ca681ce2118: Pull complete
111379f9235b: Pull complete
9db8b0fb9301: Pull complete
b2b39482e9b8: Pull complete
93f5e3ec7d59: Pull complete
38e0ece0a1d4: Pull complete
dbc7427ab4ea: Pull complete
f1cc2b939ac9: Pull complete
Digest: sha256:3df7ee55f48c08fe7dc836c60053867dac28ada8b7eb9b9413b09e5361e4937c
Status: Downloaded newer image for leearmy/cudnn6:ppc64le
u0018670@sys-90572:~$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
leearmy/cudnn6      ppc64le             de632816acda        20 hours ago        6.58 GB
ppc64le/ubuntu      latest              44c2eaa37868        3 days ago          150 MB
u0018670@sys-90572:~$





2017년 12월 10일 일요일

IBM Minsky (S822LC for HPC) 에서 nvidia-docker 설치 - Ubuntu

ppc64le 환경에서 nvidia-docker 를 설치할때마다 nvidia-docker 가 update 되고 방법이 틀려졌음.
http://hwengineer.blogspot.kr/2017/02/minsky-ubuntu-1604-xenial-ppc64le.html
https://hwengineer.blogspot.kr/2017/10/ppc64le-nvidia-docker-201710.html


2017년 12월 10일 현재 nvidia-docker 를 설치하기 위해서는 다음 사이트를 참고하였음.
https://developer.ibm.com/linuxonpower/docker-on-power/
https://nvidia.github.io/nvidia-docker/


nvidia-docker 가 설치될 환경은
-.ubuntu 16.04
-.CUDA 8.0 + cudnn 6.0.21
먼저 docker engine 설치
test_svr@ubuntu:~$
test_svr@ubuntu:~$ sudo vi /etc/apt/sources.list.d/xenial-docker.list
deb http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el/ xenial main
test_svr@ubuntu:~$ sudo apt-get update
test_svr@ubuntu:~$ sudo apt-get install docker-ce

test_svr@ubuntu:~$ dpkg -l |grep -i docker
ii  docker                                  1.5-1                                      ppc64el      System tray for KDE3/GNOME2 docklet applications
ii  docker-ce                               17.06.0~ce-0~ubuntu                        ppc64el      Docker: the open-source application container engine



docker engine 이 설치되었으면 nvidia-docker 설치
test_svr@ubuntu:~$ curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
> sudo apt-key add -
OK
test_svr@ubuntu:~$ curl -s -L https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el/nvidia-docker.list | \
>   sudo tee /etc/apt/sources.list.d/nvidia-docker.list
deb https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el /
test_svr@ubuntu:~$ sudo apt-get update
Get:1 file:/var/cuda-repo-8-0-local-ga2v2 InRelease
Ign:1 file:/var/cuda-repo-8-0-local-ga2v2 InRelease
Get:2 file:/var/nvidia-driver-local-repo-tr6 InRelease
Ign:2 file:/var/nvidia-driver-local-repo-tr6 InRelease
Get:3 file:/var/cuda-repo-8-0-local-ga2v2 Release [574 B]
Get:4 file:/var/nvidia-driver-local-repo-tr6 Release [574 B]
Get:3 file:/var/cuda-repo-8-0-local-ga2v2 Release [574 B]
Get:4 file:/var/nvidia-driver-local-repo-tr6 Release [574 B]
Get:7 http://ports.ubuntu.com/ubuntu-ports xenial-security InRelease [102 kB]
Hit:8 http://kr.ports.ubuntu.com/ubuntu-ports xenial InRelease
Get:9 https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el InRelease [1,098 B]
Get:10 http://kr.ports.ubuntu.com/ubuntu-ports xenial-updates InRelease [102 kB]
Get:11 https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el Packages [704 B]
Ign:12 http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial InRelease
Get:13 http://kr.ports.ubuntu.com/ubuntu-ports xenial-backports InRelease [102 kB]
Hit:14 http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial Release
Ign:15 http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial Release.gpg
Fetched 308 kB in 2s (111 kB/s)
Reading package lists... Done
W: The repository 'http://ftp.unicamp.br/pub/ppc64el/ubuntu/16_04/docker-17.06.0-ce-ppc64el xenial Release' is not signed.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.

test_svr@ubuntu:~$ sudo apt-get install -y nvidia-docker
 Reading package lists... Done
 Building dependency tree
 Reading state information... Done
 The following NEW packages will be installed:
 nvidia-docker
 0 upgraded, 1 newly installed, 0 to remove and 84 not upgraded.
 Need to get 2,267 kB of archives.
 After this operation, 16.0 MB of additional disk space will be used.
 Get:1 https://nvidia.github.io/nvidia-docker/ubuntu16.04/ppc64el nvidia-docker 1.0.1-1 [2,267 kB]
 Fetched 2,267 kB in 2s (964 kB/s)
 Selecting previously unselected package nvidia-docker.
 (Reading database ... 122052 files and directories currently installed.)
 Preparing to unpack .../nvidia-docker_1.0.1-1_ppc64el.deb ...
 Unpacking nvidia-docker (1.0.1-1) ...
 Processing triggers for ureadahead (0.100.0-19) ...
 ureadahead will be reprofiled on next reboot
 Setting up nvidia-docker (1.0.1-1) ...
 Configuring user
 Setting up permissions
 Processing triggers for ureadahead (0.100.0-19) ...
test_svr@ubuntu:~$

2017년 12월 7일 목요일

내가 자주 쓰는, 기억해야 하는 파이썬 command - Ubuntu

framework 버전 확인
anaconda 를 통해 pip install 로 설치된 tensorflow 1.3.1 로 동작 확인
u0018670@sys-90257:~$ env | grep PATH
PATH=/home/u0018670/bin:/home/u0018670/.local/bin:/home/u0018670/anaconda2/bin:/opt/ibm/xlC/current/bin:/opt/ibm/xlf/current/bin:/opt/at10.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
u0018670@sys-90257:~$ python
Python 2.7.13 |Anaconda, Inc.| (default, Sep 15 2017, 20:54:27)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import tensorflow as tf
>>> print(tf.__version__)
1.3.1
>>>


powerAI toolkit 에 포함된 tensorflow 로 동작

u0018670@sys-90257:~$ . /opt/DL/tensorflow/bin/tensorflow-activate
u0018670@sys-90257:~$ env | grep PATH
LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64:/usr/local/cuda-8.0/extras/CUPTI/lib64:/opt/DL/tensorflow/lib
PATH=/home/u0018670/bin:/home/u0018670/.local/bin:/home/u0018670/anaconda2/bin:/opt/ibm/xlC/current/bin:/opt/ibm/xlf/current/bin:/opt/at10.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/opt/DL/tensorflow/bin
PYTHONPATH=/opt/DL/tensorflow/lib/python2.7/site-packages
u0018670@sys-90257:~$ python
Python 2.7.13 |Anaconda, Inc.| (default, Sep 15 2017, 20:54:27)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import tensorflow as tf
>>> print(tf.__version__)
1.1.0
>>>


python system path 확인

u0018670@sys-90257:~$ python3
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/lib/python35.zip', '/usr/lib/python3.5', '/usr/lib/python3.5/plat-powerpc64le-linux-gnu', '/usr/lib/python3.5/lib-dynload', '/home/u0018670/.local/lib/python3.5/site-packages', '/usr/local/lib/python3.5/dist-packages', '/usr/lib/python3/dist-packages']
>>>


native python3 설치 후 link 생성

native python3 를 다음과 같이 설치 후,
u0018670@sys-90257:~$ sudo apt-get install python3

jupyter 도 pip install 설치
u0018670@sys-90254:~$ pip list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
bleach (2.1.1)
decorator (4.1.2)
entrypoints (0.2.3)
html5lib (1.0b10)
ipykernel (4.6.1)
ipython (6.2.1)
ipython-genutils (0.2.0)
ipywidgets (7.0.5)
jedi (0.11.0)
Jinja2 (2.10)
jsonschema (2.6.0)
jupyter (1.0.0)
jupyter-client (5.1.0)
jupyter-console (5.2.0)
jupyter-core (4.4.0)
MarkupSafe (1.0)


그런데 jupyter 가 실행이 안됨.

u0018670@sys-90257:~$ jupyter
-bash: /home/u0018670/.local/bin/jupyter: /usr/bin/python: bad interpreter: No such file or directory

안되는 이유는 /usr/bin/python 파일이 없어서 그래서 link 시켜주면 잘 됨.
u0018670@sys-90257:~$ sudo ln -s /usr/bin/python3 /usr/bin/python
u0018670@sys-90257:~$ python
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

u0018670@sys-90257:~$ jupyter
usage: jupyter [-h] [--version] [--config-dir] [--data-dir] [--runtime-dir]
               [--paths] [--json]
               [subcommand]


theano 기본 테스트
root@3498b2e65eb1:/tmp# python
Python 3.5.4 |Anaconda, Inc.| (default, Sep 19 2017, 14:42:07)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import theano
>>> import theano.tensor as T
>>> import numpy as np
>>> x = T.dscalar('x')
>>> y = x ** 2
>>> f = theano.function(inputs=[x], outputs=y)
>>> f(1)
array(1.0)
>>> f(2)
array(4.0)
>>> f(3)
array(9.0)
>>>

jupyter notebook 에서는 다음과 같이 출력 확인

2017년 12월 6일 수요일

내가 자주쓰는, 기억해야 하는 command - Ubuntu

환경변수 확인
u0018670@sys-90254:~$ more /etc/environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
u0018670@sys-90254:~$ env | grep -i path
PATH=/home/u0018670/bin:/home/u0018670/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games

설치된 패키지 경로 확인
u0018670@sys-90254:~$ dpkg -S python-pip-whl
python-pip-whl: /usr/share/doc/python-pip-whl/changelog.Debian.gz
python-pip-whl: /usr/share/doc/python-pip-whl/copyright
python-pip-whl: /usr/share/doc/python-pip-whl
u0018670@sys-90254:~$ dpkg -S ifconfig
net-tools: /sbin/ifconfig
net-tools: /usr/share/man/de/man8/ifconfig.8.gz
linux-headers-4.4.0-103: /usr/src/linux-headers-4.4.0-103/tools/hv/hv_set_ifconfig.sh
net-tools: /usr/share/man/man8/ifconfig.8.gz
net-tools: /usr/share/man/pt_BR/man8/ifconfig.8.gz
linux-headers-4.4.0-104: /usr/src/linux-headers-4.4.0-104/tools/hv/hv_set_ifconfig.sh
net-tools: /usr/share/man/fr/man8/ifconfig.8.gz


/opt/at10.0 경로
IBM 패키지 중 advance-toolchain, xlc 등이 설치되는 경로 여기에 pip3 도 있고 gcc 도 있고... 과감히 삭제
u0018670@sys-90257:/opt/at10.0/bin$ dpkg -l |grep advance
ii  advance-toolchain-at10.0-devel      10.0-2                                     ppc64el      Advance Toolchain
ii  advance-toolchain-at10.0-mcore-libs 10.0-2                                     ppc64el      Advance Toolchain
ii  advance-toolchain-at10.0-perf       10.0-2                                     ppc64el      Advance Toolchain
ii  advance-toolchain-at10.0-runtime    10.0-2                                     ppc64el      Advance Toolchain

2017년 12월 5일 화요일

Python 2.7 과 Python 3.5 동시 사용 - Ubuntu

Anaconda2 로 python 2.7 이 설치된 환경에서 root 에 영향이 없이, 다시말해 Anaconda3 를 설치하지 않아도 python 3.5 를 사용하기 위해 다음과 같이 수행

현재 설치된 anaconda2 확인
u0018670@sys-90257:~$ which python
/home/u0018670/anaconda2/bin/python

jupyter kernal 확인
u0018670@sys-90257:~$ jupyter kernelspec list
Available kernels:
  python2    /home/u0018670/anaconda2/share/jupyter/kernels/python2

jupyter notebook 을 실행해봐도 Python2 만 확인

u0018670@sys-90257:~$ conda update conda
Fetching package metadata ...........
Solving package specifications: .

# All requested packages already installed.
# packages in environment at /home/u0018670/anaconda2:
#
conda                     4.3.27                   py27_0

python 3.5 환경으로 업데이트
u0018670@sys-90257:~$ conda create -n py35 python=3.5 anaconda
Fetching package metadata ...........
Solving package specifications: .

Package plan for installation in environment /home/u0018670/anaconda2/envs/py35:

The following NEW packages will be INSTALLED:

    alabaster:                0.7.10-py35_0

    anaconda:                 5.0.0-py35h419c926_0
    anaconda-client:          1.6.3-py35_0
    anaconda-project:         0.6.0-py35_0
    asn1crypto:               0.22.0-py35_0
    astroid:                  1.5.3-py35_0
    astropy:                  2.0.1-np113py35_1
    babel:                    2.5.0-py35_0
    backports:                1.0-py35_0
    beautifulsoup4:           4.6.0-py35_0
    bitarray:                 0.8.1-py35_0
......................
Proceed ([y]/n)? y

python-3.5.4-0 100% |#####################################################################################| Time: 0:00:01  14.46 MB/s
alabaster-0.7. 100% |#####################################################################################| Time: 0:00:00  15.39 MB/s
asn1crypto-0.2 100% |#####################################################################################| Time: 0:00:00  16.71 MB/s
backports-1.0- 100% |#####################################################################################| Time: 0:00:00   3.98 MB/s
beautifulsoup4 100% |#####################################################################################| Time: 0:00:00  18.22 MB/s
bitarray-0.8.1 100% |#####################################################################################| Time: 0:00:00  25.68 MB/s
boto-2.48.0-py 100% |#####################################################################################| Time: 0:00:00   3.83 MB/s
......................
anaconda-5.0.0 100% |#####################################################################################| Time: 0:00:00  12.86 MB/s
#
# To activate this environment, use:
# > source activate py35
#
# To deactivate an active environment, use:
# > source deactivate
#
u0018670@sys-90257:~$ source activate py35
(py35) u0018670@sys-90257:~$ jupyter notebook

(py35) u0018670@sys-90257:~/anaconda2$ which python
/home/u0018670/anaconda2/envs/py35/bin/python

(py35) u0018670@sys-90257:~/anaconda2$ source deactivate
u0018670@sys-90257:~/anaconda2$ 


추가적으로, 특정 버전의 python 패키지를 설치할때는 다음과 같이
그냥 keras 설치를 해버리면 최신 버전이 설치됨.
nimbix@JARVICENAE-0A0A183D:~$ pip install keras
Collecting keras
  Downloading Keras-2.1.2-py2.py3-none-any.whl (304kB)
    100% |████████████████████████████████| 307kB 4.3MB/s
Requirement already satisfied: scipy>=0.14 in ./anaconda3/lib/python3.6/site-packages (from keras)
Requirement already satisfied: six>=1.9.0 in ./anaconda3/lib/python3.6/site-packages (from keras)
Requirement already satisfied: pyyaml in ./anaconda3/lib/python3.6/site-packages (from keras)
Requirement already satisfied: numpy>=1.9.1 in ./anaconda3/lib/python3.6/site-packages (from keras)
Installing collected packages: keras
Successfully installed keras-2.1.2
 

그런데, 
nimbix@JARVICENAE-0A0A183D:~$ pip install keras==1.2
Collecting keras==1.2
  Downloading Keras-1.2.0.tar.gz (167kB)
    100% |████████████████████████████████| 174kB 6.4MB/s
Collecting theano (from keras==1.2)
  Downloading Theano-1.0.1.tar.gz (2.8MB)
    100% |████████████████████████████████| 2.8MB 506kB/s
Requirement already satisfied: pyyaml in ./anaconda3/lib/python3.6/site-packages (from keras==1.2)
Requirement already satisfied: six in ./anaconda3/lib/python3.6/site-packages (from keras==1.2)
Requirement already satisfied: numpy>=1.9.1 in ./anaconda3/lib/python3.6/site-packages (from theano->keras==1.2)
Requirement already satisfied: scipy>=0.14 in ./anaconda3/lib/python3.6/site-packages (from theano->keras==1.2)
Building wheels for collected packages: keras, theano
  Running setup.py bdist_wheel for keras ... done
  Stored in directory: /home/nimbix/.cache/pip/wheels/f8/2c/8e/27d952128220b1acf6fcd5e692269c6e8d60744c15fd160c2c
  Running setup.py bdist_wheel for theano ... done
  Stored in directory: /home/nimbix/.cache/pip/wheels/46/a2/7d/b4cac381d5151daa9f9e0b3e4e4b65edaea6355ae296c97cf2
Successfully built keras theano
Installing collected packages: theano, keras
  Found existing installation: Keras 2.1.2
    Uninstalling Keras-2.1.2:
      Successfully uninstalled Keras-2.1.2
Successfully installed keras-1.2.0 theano-1.0.1
 

기존 2.1.2 를 자동으로 삭제하고 1.2 가 설치됨. 

2017년 12월 4일 월요일

폐쇄망에서 python 패키지 설치 - Ubuntu

먼저, 인터넷이 되는 곳에서 원하는 패키지를 다음과 같이 다운로드 받는다.

online@ubuntu:~$ pip install --download . tensorflow
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting tensorflow
  Downloading tensorflow-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl (40.7MB)
    100% |████████████████████████████████| 40.8MB 23kB/s
  Saved ./tensorflow-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting enum34>=1.1.6 (from tensorflow)
  Downloading enum34-1.1.6-py2-none-any.whl
  Saved ./enum34-1.1.6-py2-none-any.whl
Collecting backports.weakref>=1.0rc1 (from tensorflow)
  Downloading backports.weakref-1.0.post1-py2.py3-none-any.whl
  Saved ./backports.weakref-1.0.post1-py2.py3-none-any.whl
Collecting wheel (from tensorflow)
  Downloading wheel-0.30.0-py2.py3-none-any.whl (49kB)
    100% |████████████████████████████████| 51kB 6.9MB/s
  Saved ./wheel-0.30.0-py2.py3-none-any.whl
Collecting mock>=2.0.0 (from tensorflow)
  Downloading mock-2.0.0-py2.py3-none-any.whl (56kB)
    100% |████████████████████████████████| 61kB 2.7MB/s
  Saved ./mock-2.0.0-py2.py3-none-any.whl
Collecting tensorflow-tensorboard<0.5.0,>=0.4.0rc1 (from tensorflow)
  Downloading tensorflow_tensorboard-0.4.0rc3-py2-none-any.whl (1.7MB)
    100% |████████████████████████████████| 1.7MB 537kB/s
  Saved ./tensorflow_tensorboard-0.4.0rc3-py2-none-any.whl
Collecting numpy>=1.12.1 (from tensorflow)
  Downloading numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl (16.6MB)
    100% |████████████████████████████████| 16.7MB 66kB/s
  Saved ./numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Collecting protobuf>=3.3.0 (from tensorflow)
  Downloading protobuf-3.5.0.post1-cp27-cp27mu-manylinux1_x86_64.whl (6.4MB)
    100% |████████████████████████████████| 6.4MB 166kB/s
  Saved ./protobuf-3.5.0.post1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting six>=1.10.0 (from tensorflow)
  Downloading six-1.11.0-py2.py3-none-any.whl
  Saved ./six-1.11.0-py2.py3-none-any.whl
Collecting funcsigs>=1; python_version < "3.3" (from mock>=2.0.0->tensorflow)
  Downloading funcsigs-1.0.2-py2.py3-none-any.whl
  Saved ./funcsigs-1.0.2-py2.py3-none-any.whl
Collecting pbr>=0.11 (from mock>=2.0.0->tensorflow)
  Downloading pbr-3.1.1-py2.py3-none-any.whl (99kB)
    100% |████████████████████████████████| 102kB 1.6MB/s
  Saved ./pbr-3.1.1-py2.py3-none-any.whl
Collecting bleach==1.5.0 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
  Downloading bleach-1.5.0-py2.py3-none-any.whl
  Saved ./bleach-1.5.0-py2.py3-none-any.whl
Collecting futures>=3.1.1; python_version < "3.2" (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
  Downloading futures-3.2.0-py2-none-any.whl
  Saved ./futures-3.2.0-py2-none-any.whl
Collecting markdown>=2.6.8 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
  Downloading Markdown-2.6.9.tar.gz (271kB)
    100% |████████████████████████████████| 276kB 1.3MB/s
  Saved ./Markdown-2.6.9.tar.gz
Collecting werkzeug>=0.11.10 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
  Downloading Werkzeug-0.12.2-py2.py3-none-any.whl (312kB)
    100% |████████████████████████████████| 317kB 1.3MB/s
  Saved ./Werkzeug-0.12.2-py2.py3-none-any.whl
Collecting html5lib==0.9999999 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
  Downloading html5lib-0.9999999.tar.gz (889kB)
    100% |████████████████████████████████| 890kB 744kB/s
  Saved ./html5lib-0.9999999.tar.gz
Collecting setuptools (from protobuf>=3.3.0->tensorflow)
  Downloading setuptools-38.2.4-py2.py3-none-any.whl (489kB)
    100% |████████████████████████████████| 491kB 1.0MB/s
  Saved ./setuptools-38.2.4-py2.py3-none-any.whl
Successfully downloaded tensorflow enum34 backports.weakref wheel mock tensorflow-tensorboard numpy protobuf six funcsigs pbr bleach futures markdown werkzeug html5lib setuptools


다운로드 받은 패키지들은 의존성에 의해 관련된 패키지들도 함께 다운받아지며 이것들을 폐쇄망 서버로 이동(tensorflow 디렉토리) 후, 다음과 같이 설치.

offline@ubuntu:~/tensorflow$ pip install --no-index --find-links=. tensorflow
Collecting tensorflow
Requirement already satisfied: enum34>=1.1.6 in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow)
Collecting backports.weakref>=1.0rc1 (from tensorflow)
Requirement already satisfied: wheel in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow)
Collecting mock>=2.0.0 (from tensorflow)
Collecting tensorflow-tensorboard<0.5.0,>=0.4.0rc1 (from tensorflow)
Requirement already satisfied: numpy>=1.12.1 in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow)
Collecting protobuf>=3.3.0 (from tensorflow)
Requirement already satisfied: six>=1.10.0 in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/testuser01/anaconda2/lib/python2.7/site-packages (from mock>=2.0.0->tensorflow)
Collecting pbr>=0.11 (from mock>=2.0.0->tensorflow)
Collecting bleach==1.5.0 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Requirement already satisfied: futures>=3.1.1; python_version < "3.2" in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Collecting markdown>=2.6.8 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Requirement already satisfied: werkzeug>=0.11.10 in /home/testuser01/anaconda2/lib/python2.7/site-packages (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Collecting html5lib==0.9999999 (from tensorflow-tensorboard<0.5.0,>=0.4.0rc1->tensorflow)
Requirement already satisfied: setuptools in /home/testuser01/anaconda2/lib/python2.7/site-packages (from protobuf>=3.3.0->tensorflow)
Building wheels for collected packages: markdown, html5lib
  Running setup.py bdist_wheel for markdown ... done
  Stored in directory: /home/testuser01/.cache/pip/wheels/51/36/90/47aaba211d675deb9af500b7a894541103b283c5f18e0ef529
  Running setup.py bdist_wheel for html5lib ... done
  Stored in directory: /home/testuser01/.cache/pip/wheels/22/7a/d3/e96250b678c5b3af1d50336781a06e05568d81251c10440b4a
Successfully built markdown html5lib
Installing collected packages: backports.weakref, pbr, mock, html5lib, bleach, protobuf, markdown, tensorflow-tensorboard, tensorflow
  Found existing installation: html5lib 0.999999999
    Uninstalling html5lib-0.999999999:
      Successfully uninstalled html5lib-0.999999999
  Found existing installation: bleach 2.0.0
    Uninstalling bleach-2.0.0:
      Successfully uninstalled bleach-2.0.0
Successfully installed backports.weakref-1.0.post1 bleach-1.5.0 html5lib-0.9999999 markdown-2.6.9 mock-2.0.0 pbr-3.1.1 protobuf-3.5.0.post1 tensorflow-1.4.0 tensorflow-tensorboard-0.4.0rc3
offline@ubuntu:~/tensorflow$ conda list | grep tensorflow
tensorflow                1.4.0                     <pip>
tensorflow-tensorboard    0.4.0rc3                  <pip>

2017년 12월 3일 일요일

IBM Minsky (S822LC for HPC) 에서 tensorflow 1.3 사용 - Ubuntu

IBM GPU 서버에서 tensorflow 1.3 을 사용하기 위해서는 source code 로 build 를 해야 하며, 다음 link 를 참고.

https://hwengineer.blogspot.kr/2017/10/minsky-tensorflow-r13-source-build.html

위 link 에서는 anaconda 디렉토리 내에 packages 디렉토리만 덮어쓰면 된다는데 안되는 경우에는 anaconda 디렉토리를 통째로 덮어쓰면 잘 됨.

다음 link 에anaconda 디렉토리를 통째로 압축한 파일을 다운 받을 수 있음.
참고로, root 권한으로 anaconda 를 설치하였고, 버전은 2.7 

https://drive.google.com/file/d/1kruGMjrxcryhnTtuBMh6gRU_zgAaDU2d/view?usp=sharing

2017년 11월 20일 월요일

deb package 삭제 - Ubuntu

Ubuntu 에서 설치된 package(.deb) 를 삭제

기존 설치된 cuda 9.0 삭제를 예시
u0018670@sys-89985:~$ dpkg -l |grep cuda
ri  cuda-9-0                               9.0.176-1                                  ppc64el      CUDA 9.0 meta-package
ii  cuda-command-line-tools-9-0            9.0.176-1                                  ppc64el      CUDA command-line tools
ii  cuda-core-9-0                          9.0.176-1                                  ppc64el      CUDA core tools
ii  cuda-cublas-8-0                        8.0.61.2-1                                 ppc64el      CUBLAS native runtime libraries
ii  cuda-cublas-9-0                        9.0.176-1                                  ppc64el      CUBLAS native runtime libraries
ii  cuda-cublas-dev-9-0                    9.0.176-1                                  ppc64el      CUBLAS native dev links, headers
ii  cuda-cudart-8-0                        8.0.61-1                                   ppc64el      CUDA Runtime native Libraries
ii  cuda-cudart-9-0                        9.0.176-1                                  ppc64el      CUDA Runtime native Libraries
ii  cuda-cudart-dev-9-0                    9.0.176-1                                  ppc64el      CUDA Runtime native dev links, headers
ii  cuda-cufft-8-0                         8.0.61-1                                   ppc64el      CUFFT native runtime libraries
ii  cuda-cufft-9-0                         9.0.176-1                                  ppc64el      CUFFT native runtime libraries
ii  cuda-cufft-dev-9-0                     9.0.176-1                                  ppc64el      CUFFT native dev links, headers
...



u0018670@sys-89985:~$ sudo apt-get --purge remove cuda
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package 'cuda' is not installed, so not removed
The following packages were automatically installed and are no longer required:
  acpid bbswitch-dkms bumblebee cuda-9-0 cuda-command-line-tools-9-0 cuda-core-9-0 cuda-cublas-9-0 cuda-cublas-dev-9-0
  cuda-cudart-9-0 cuda-cudart-dev-9-0 cuda-cufft-9-0 cuda-cufft-dev-9-0 cuda-curand-9-0 cuda-curand-dev-9-0 cuda-cusolver-9-0
  cuda-cusolver-dev-9-0 cuda-cusparse-9-0 cuda-cusparse-dev-9-0 cuda-documentation-9-0 cuda-driver-dev-9-0 cuda-drivers
  cuda-libraries-9-0 cuda-libraries-dev-9-0 cuda-license-9-0 cuda-misc-headers-9-0 cuda-npp-9-0 cuda-npp-dev-9-0 cuda-nvgraph-9-0
  cuda-nvgraph-dev-9-0 cuda-nvml-dev-9-0 cuda-nvrtc-9-0 cuda-nvrtc-dev-9-0 cuda-runtime-9-0 cuda-samples-9-0 cuda-toolkit-9-0
  cuda-visual-tools-9-0 dkms libcuda1-384 libjansson4 libxfont1 libxkbfile1 libxnvctrl0 nvidia-384 nvidia-384-dev nvidia-modprobe
  nvidia-settings policykit-1-gnome python3-xkit screen-resolution-extra x11-xkb-utils xfonts-base xfonts-encodings xfonts-utils
  xserver-common xserver-xorg-core xserver-xorg-legacy
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 86 not upgraded.


다음 command 를 통해 삭제 수행.
u0018670@sys-89985:~$ sudo apt autoremove
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
  acpid bbswitch-dkms bumblebee cuda-9-0 cuda-command-line-tools-9-0 cuda-core-9-0 cuda-cublas-9-0 cuda-cublas-dev-9-0
  cuda-cudart-9-0 cuda-cudart-dev-9-0 cuda-cufft-9-0 cuda-cufft-dev-9-0 cuda-curand-9-0 cuda-curand-dev-9-0 cuda-cusolver-9-0
  cuda-cusolver-dev-9-0 cuda-cusparse-9-0 cuda-cusparse-dev-9-0 cuda-documentation-9-0 cuda-driver-dev-9-0 cuda-drivers
  cuda-libraries-9-0 cuda-libraries-dev-9-0 cuda-license-9-0 cuda-misc-headers-9-0 cuda-npp-9-0 cuda-npp-dev-9-0 cuda-nvgraph-9-0
  cuda-nvgraph-dev-9-0 cuda-nvml-dev-9-0 cuda-nvrtc-9-0 cuda-nvrtc-dev-9-0 cuda-runtime-9-0 cuda-samples-9-0 cuda-toolkit-9-0
  cuda-visual-tools-9-0 dkms libcuda1-384 libjansson4 libxfont1 libxkbfile1 libxnvctrl0 nvidia-384 nvidia-384-dev nvidia-modprobe
  nvidia-settings policykit-1-gnome python3-xkit screen-resolution-extra x11-xkb-utils xfonts-base xfonts-encodings xfonts-utils
  xserver-common xserver-xorg-core xserver-xorg-legacy
0 upgraded, 0 newly installed, 56 to remove and 85 not upgraded.
After this operation, 2,236 MB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 190184 files and directories currently installed.)
Removing cuda-9-0 (9.0.176-1) ...
Removing cuda-runtime-9-0 (9.0.176-1) ...
Removing cuda-drivers (384.81-1) ...
Removing nvidia-384-dev (384.81-0ubuntu1) ...
...
u0018670@sys-89985:~$ sudo apt-get clean


추가로, CUDA 8.0 을 설치하고 싶은데 NVIDIA 사이트에는 최신인 CUDA 9.0 링크 제공
https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=ppc64le&target_distro=Ubuntu&target_version=1604&target_type=debnetwork

그냥 apt-get 으로 설치.
u0018670@sys-89985:~$ sudo apt-get install cuda-8-0
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  acpid bbswitch-dkms bumblebee cuda-command-line-tools-8-0 cuda-core-8-0 cuda-cublas-dev-8-0 cuda-cudart-dev-8-0
  cuda-cufft-dev-8-0 cuda-curand-dev-8-0 cuda-cusolver-dev-8-0 cuda-cusparse-dev-8-0 cuda-documentation-8-0 cuda-driver-dev-8-0
  cuda-drivers cuda-misc-headers-8-0 cuda-npp-8-0 cuda-npp-dev-8-0 cuda-nvgraph-8-0 cuda-nvgraph-dev-8-0 cuda-nvml-dev-8-0
  cuda-nvrtc-8-0 cuda-nvrtc-dev-8-0 cuda-runtime-8-0 cuda-samples-8-0 cuda-toolkit-8-0 cuda-visual-tools-8-0 dkms libcuda1-384
  libjansson4 libxfont1 libxkbfile1 libxnvctrl0 nvidia-384 nvidia-384-dev nvidia-modprobe nvidia-settings policykit-1-gnome
  python3-xkit screen-resolution-extra x11-xkb-utils xfonts-base xfonts-encodings xfonts-utils xserver-common xserver-xorg-core
  xserver-xorg-legacy
Suggested packages:
......


아니면, 다음 link 를 통해 다운로드
https://developer.nvidia.com/cuda-toolkit-archive 

2017년 11월 19일 일요일

NFS 서버, 클라이언트 구성 - Ubuntu

NFS 서버(10.10.10.10), NFS 클라이언트(20.20.20.20)

서버(10.10.10.10)에서 NFS 관련 package 설치
root@ubuntu:~#apt-get install nfs-common nfs-kernel-server rpcbind

/data/nfs 디렉토리 생성 후 chmod 777 권한 변경
root@ubuntu:/data$ ls -al
total 28
drwxr-xr-x  4 root root  4096 Nov 19 21:41 .
drwxr-xr-x 24 root root  4096 Nov 19 18:02 ..
drwx------  2 root root 16384 Nov 19 18:02 lost+found
drwxrwxrwx  2 root root  4096 Nov 19 21:41 nfs


nfs 설정 파일에 공유할 폴더와 설정 추가
root@ubuntu:~#vi /etc/exports
# /etc/exports: the access control list for filesystems which may be exported
#               to NFS clients.  See exports(5).
#
# Example for NFSv2 and NFSv3:
# /srv/homes       hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
#
# Example for NFSv4:
# /srv/nfs4        gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes  gss/krb5i(rw,sync,no_subtree_check)
#
/data/nfs       *(rw,sync,no_subtree_check)


root@ubuntu:~#service nfs-kernel-server restart
root@ubuntu:~#service rpcbind stop
root@ubuntu:~#service rpcbind start

클라이언트(20.20.20.20)에서 nfs-common 패키지 설
root@ubuntu:~#apt-get install nfs-common

서버에서 export 된 디렉토리 /root/v7000 에 mount
root@ubuntu:~# mount -t nfs -o nolock 10.10.10.10:/data/nfs /root/v7000
root@ubuntu:~/v7000# df -h
Filesystem               Size  Used Avail Use% Mounted on
udev                     475M     0  475M   0% /dev
tmpfs                     99M  4.8M   95M   5% /run
/dev/sda1                 19G   11G  7.4G  59% /
tmpfs                    495M  4.0K  495M   1% /dev/shm
tmpfs                    5.0M     0  5.0M   0% /run/lock
tmpfs                    495M     0  495M   0% /sys/fs/cgroup
tmpfs                     99M     0   99M   0% /run/user/0
10.10.10.10:/data/nfs   50G   52M   47G   1% /root/v7000

iSCSI 스토리지 디스크 연결 - Ubuntu

설치 환경
root@ubuntu:~# more /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.2 LTS"


먼저, iSCSI initiator 설치
root@ubuntu:~# sudo apt-get install open-iscsi
root@ubuntu:~# dpkg -l |grep iscsi
ii  open-iscsi                         2.0.873+git0.3b4b4500-14ubuntu3.3  amd64        iSCSI initiator tools


initiator 설치되면 다음과 같이 디렉토리에서 iqn 정보 확인 가능
root@ubuntu:/etc/iscsi# pwd
/etc/iscsi
root@ubuntu:/etc/iscsi# ls -al
total 32
drwxr-xr-x  4 root root  4096 Nov 19 17:27 .
drwxr-xr-x 89 root root  4096 Nov 19 17:06 ..
-rw-------  1 root root   349 Nov  7 21:03 initiatorname.iscsi
-rw-------  1 root root 11788 Nov 19 17:27 iscsid.conf
drw-------  4 root root  4096 Nov 19 17:32 nodes
drw-------  4 root root  4096 Nov  8 16:12 send_targets
root@ubuntu:/etc/iscsi# more initiatorname.iscsi
## DO NOT EDIT OR REMOVE THIS FILE!
## If you remove this file, the iSCSI daemon will not start.
## If you change the InitiatorName, existing access control lists
## may reject this initiator.  The InitiatorName must be unique
## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.
InitiatorName=iqn.1993-08.org.debian:01:aeff61f6e0e3


iscsid.conf 파일을 다음과 같이 수정

이제 스토리지(Storwize IBM V7000)에서 iSCSI 환경 설정, 볼륨 생성 및 해당 서버에 볼륨 할당

iSCSI 볼륨을 사용하기 위해서는 네트워크 - 이더넷 포트 메뉴에서 IP 설정을 해주어야 한다. 당연히 V7000 GUI 접속을 위한 management IP 와는 다른 IP를 설정하는데 UTP 케이블은 물리적으로 동일한 port 를 사용한다.

스토리지에서 호스트 생성시 서버의 iqn 정보를 입력하고 생성한다.

스토리지 볼륨(50GB) 생성된 호스트에 할당


이제 다시 서버에서 스토리지 정보 인식. 스토리지에서 설정한 iSCSI 용 IP 입력
root@ubuntu:~# iscsiadm -m discovery -t st -p 9.187.227.230
9.187.227.230:3260,1 iqn.1986-03.com.ibm:2145.stg-v7000.node1
root@ubuntu:~# iscsiadm -m discovery -t st -p 9.187.227.240
9.187.227.240:3260,1 iqn.1986-03.com.ibm:2145.stg-v7000.node2



스토리지 연결. 다음과 같이 successful 이 나오면 성공
root@ubuntu:~# iscsiadm -m node --login
Logging in to [iface: default, target: iqn.1986-03.com.ibm:2145.stg-v7000.node2, portal: 9.187.227.240,3260] (multiple)
Logging in to [iface: default, target: iqn.1986-03.com.ibm:2145.stg-v7000.node1, portal: 9.187.227.230,3260] (multiple)
Login to [iface: default, target: iqn.1986-03.com.ibm:2145.stg-v7000.node2, portal: 9.187.227.240,3260] successful.
Login to [iface: default, target: iqn.1986-03.com.ibm:2145.stg-v7000.node1, portal: 9.187.227.230,3260] successful.


그리고, dmesg 로 연결된 스토리지 볼륨 확인
root@ubuntu:~# dmesg | tail -100 | grep sd
[    5.273551] EXT4-fs (sda1): re-mounted. Opts: errors=remount-ro
[    7.079363] Adding 4191228k swap on /dev/sda5.  Priority:-1 extents:1 across:4191228k FS
[597769.393140] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 39s! [vmtoolsd:985]
[597769.569910] CPU: 0 PID: 985 Comm: vmtoolsd Not tainted 4.4.0-62-generic #83-Ubuntu
[953286.712454] sd 4:0:0:0: [sdb] 104857600 512-byte logical blocks: (53.7 GB/50.0 GiB)
[953286.732047] sd 4:0:0:0: [sdb] Write Protect is off
[953286.732051] sd 4:0:0:0: [sdb] Mode Sense: 97 00 10 08
[953286.738440] sd 4:0:0:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
[953286.738663] sd 3:0:0:0: [sdc] 104857600 512-byte logical blocks: (53.7 GB/50.0 GiB)
[953286.746023] sd 3:0:0:0: [sdc] Write Protect is off
[953286.746027] sd 3:0:0:0: [sdc] Mode Sense: 97 00 10 08
[953286.746522] sd 3:0:0:0: [sdc] Write cache: disabled, read cache: enabled, supports DPO and FUA
[953286.808293] sd 3:0:0:0: Attached scsi generic sg2 type 0
[953286.808727] sd 4:0:0:0: Attached scsi generic sg3 type 0
[953286.857392] sd 3:0:0:0: [sdc] Attached SCSI disk
[953286.857563] sd 4:0:0:0: [sdb] Attached SCSI disk


스토리지의 컨트롤러가 2개이여서 sdb, sdc 로 볼륨 확인.

이제 multipath 구성을 위해 multipath-tool package 설치
root@ubuntu:~# apt-get install multipath-tools

설치후 다음과 같이 스토리지 볼륨을 확인 할 수 있으나,
root@ubuntu:~# multipath -ll
36005076802820c032000000000000055 dm-0 IBM,2145
size=50G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| `- 3:0:0:0 sdc 8:32 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  `- 4:0:0:0 sdb 8:16 active ready running


/etc/multipath.conf 파일을 수정하여 볼륨을 alias 정의 후 사용. 기본적으로 multipath.conf 을 없으니 vi 로 파일 생성 후 다음과 같이 수정.
multipaths {
multipath {
wwid 36005076802820c032000000000000055
alias vol01
path_grouping_policy multibus
path_selector "round-robin 0"
failback manual
rr_weight priorities
no_path_retry 5
}
}


multipath 의 자세한 내용은 다음 링크 또는 각 스토리지 벤더에서 제공하는 방법을 참고.
https://help.ubuntu.com/lts/serverguide/multipath-dm-multipath-config-file.html

root@ubuntu:~# service multipathd restart
root@ubuntu:~# multipath -ll
vol01 (36005076802820c032000000000000055) dm-0 IBM,2145
size=50G features='1 queue_if_no_path' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=30 status=active
  |- 3:0:0:0 sdc 8:32 active ready running
  `- 4:0:0:0 sdb 8:16 active ready running


생성된 볼륨 포멧 및 마운트
root@ubuntu:~# fdisk -l |grep -i vol01
Disk /dev/mapper/vol01: 50 GiB, 53687091200 bytes, 104857600 sectors

root@ubuntu:~# mkfs.ext4 /dev/mapper/vol01
mke2fs 1.42.13 (17-May-2015)
Creating filesystem with 13107200 4k blocks and 3276800 inodes
Filesystem UUID: 3a2d7e7d-580a-4fd1-baaf-66bdb688ecb0
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done


root@ubuntu:~# mount /dev/mapper/vol01 /data/
root@ubuntu:~# df -h
Filesystem         Size  Used Avail Use% Mounted on
udev               2.0G     0  2.0G   0% /dev
tmpfs              396M  5.8M  390M   2% /run
/dev/sda1           26G  2.0G   23G   9% /
tmpfs              2.0G     0  2.0G   0% /dev/shm
tmpfs              5.0M     0  5.0M   0% /run/lock
tmpfs              2.0G     0  2.0G   0% /sys/fs/cgroup
tmpfs              396M     0  396M   0% /run/user/1000
/dev/mapper/vol01   50G   52M   47G   1% /data