First Try of AWS

First try of Amazon AWS, as in course Natural Language Processing. I should have reviewed this tutorial a week ago! Use all the time I spent on Facebook and Douban and talking to people…

Step-by-step guidance from the course tutor. I will note down my understandings and relevant materials in red.

AWS

    • Sign in to AWS
    • Under Services tab on the upper left corner, click EC2 under Compute section
    • On the upper right corner, switch your region to Oregon.
    • Under IMAGES section click AMIs
    • On the dropdown box in the search bar, change from “Owned by me” to “Public images
    • To launch a CPU EC2 Instance with PyTorch environment
      • Search for NYU-DSGA1011-PyTorch-CPU-0
      • Right click the AMI and click Launch
      • Select your instance type (t2.micro for instance)
      • Click Review and Launch
      • In the pop-up window, select Choose an existing key pair and select your key pair below
      • Need to create my own key pair before this. What is a key pair? http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#having-ec2-create-your-key-pair
      • So seems to just be a key / username/password. My keypair for this is “nlp2017” and the file is “nlp2017.pem.txt”
      • Click View Instance
      • Here instances need to be in a running state until it can be connected! How to connect here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AccessingInstances.html?icmpid=docs_ec2_console
      • Open a terminal window, type “ssh -i [my-key-pair.pem] ec2-user@[dns]”. Please replace [my-key-pair.pem] with a directory to your key pair and [dns] with the string under Public DNS (IPv4) column.
        • chmod 600 nlp2017.pem.txt [This is to change permission see this thread: https://stackoverflow.com/questions/9270734/ssh-permissions-are-too-open-error]ssh –i nlp2017.pem.txt ec2-user@ec2-52-37-234-187.us-west-2.compute.amazonaws.com
      • source ~/.bashrc
      • cd pytorch_test/src/
      • python pytorch_test_lr_cpu.py
      • You can confirm it’s working by observing messages like “Epoch: [1/5], Step: [100/600], Loss: 2.2161”
      • Example client code from https://github.com/yunjey/pytorch-tutorial/blob/master/tutorials/01-basics/logistic_regression/main.py
      • In the future, you can replace the example client code with your .py file.
    • To launch a GPU EC2 Instance with PyTorch environment.
    • Remember to stop an instance ! http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Stop_Start.html

 

  • You will be charged for any EC2 Instance with GPU

 

      • Search for nyu-nlp-pytorch-gpu-0
      • Right click the AMI and click Launch
      • Select g2.2xlarge as your instance type
      • Click Review and Launch
      • In the pop-up window, select Choose an existing key pair and select your key pair below
      • Click View Instance
      • Open a terminal window, type “ssh -i [my-key-pair.pem] ec2-user@[dns]”. Please replace [my-key-pair.pem] with a directory to your key pair and [dns] with the string under Public DNS (IPv4) column.
      • source ~/.bashrc
      • cd nlp_client_code/src/
      • python pytorch_cnn_tutorial_gpu.py
      • You can confirm it’s working by observing messages like “Epoch [1/5], Iter [100/600] Loss: 0.2209”
      • Example client code from https://github.com/yunjey/pytorch-tutorial/blob/master/tutorials/02-intermediate/convolutional_neural_network/main-gpu.py
      • In the future, you can replace the example client code with your .py file.

 

  • Please don’t forget to stop (or terminate) your instance. Otherwise, you may be charged for AWS usage.

 

 

Share this post

Leave a Reply

Your email address will not be published. Required fields are marked *