在线时间:8:00-16:00
迪恩网络APP
随时随地掌握行业动态
扫描二维码
关注迪恩网络微信公众号
开源软件名称(OpenSource Name):cs231n/gcloud开源软件地址(OpenSource Url):https://github.com/cs231n/gcloud开源编程语言(OpenSource Language):Python 99.2%开源软件介绍(OpenSource Introduction):Google Cloud Setup and Tutorial(Last Update on April 7, 2021) Table of contents
OverviewFor your class project, we recommend setting up a GPU instance on GCP (Google Cloud Platform). BIG REMINDER: Make sure you stop your instances!(We know you won't read until the very bottom once your assignment is running, so we are printing this at the top too since it is super important) Don't forget to stop your instance when you are done (by clicking on the stop button at the top of the page showing your instances), otherwise you will run out of credits and that will be very sad. :( If you follow our instructions below correctly, you should be able to restart your instance and the downloaded software will still be available. Colab vs GCPWhile Colab is good for assignments, and is still a helpful and free tool for experimentation for your project, you will likely need a dedicated GPU instance when you start training on large datasets and collaborating as a team:
Create and Configure Your AccountYou should use your personal GMail account for GCP, i.e. NOT [email protected], because Stanford University managed email accounts do not support creating a new project. For the class project, we offer students $50 GCP coupons for each student to use Google Compute Engine for developing and testing your implementations. When you first sign up on GCP, you will have $300 free credits. If your credits ends up not being enough, contact course staff on Piazza. We will also send out forms for extra GCP credit request form later in the quarter. This tutorial lists the necessary steps of working on the projects using Google Cloud. We expect this tutorial to take up to an hour. Don't get intimidated by the steps, we tried to make the tutorial detailed so that you are less likely to get stuck on a particular step. Please tag all questions related to Google Cloud with google_cloud on Piazza. Sign Up GCP for the First TimeYou should receive $300 credits from Google when you first sign up with Personal GMail and also UPGRADE it into a full account. Please try to use the resources judiciously.
Configure Your Project
Claim CS231N GCP creditsNOTE: You should have created and logged in your GCP account registered with your personal gmail account by now.
i.e. You can use up the $300 free credits first. Then switch to the CS231N billing account referring to this GCloud documentation. Request an Increase in GPU QuotaYour account typically does not come with GPU quota. You have to explicitly request for it under IAM Admin > Quotas. Please request the quota increase ASAP, because they will take up between couple minutes to a week to process! If you don't have GPU quota, you will have to create a CPU-only VM first and create another GPU VM later, explained in the next section. You will need to change your quota for GPU (all regions).
Why don't I See Any GPU-related Quota
One way I found that can make Compute Engine API setup faster is by visiting the VM instance page by clicking Compute Engine > VM instances If you see that Compute Engine is not ready yet, wait for couple minutes until you see something like this screenshot below. The GPU-related Quota should now show up in IAM Admin > Quotas.
More instructions at General quota instructions and Step-by-step GPU-specific walk-through (all answers in the link are useful) Set Up Google Cloud VM ImageCustomize VM Hardware
Your configuration sheet should look similar to below image. Follow exactly the same configuration for the ones with red boxes. For configurations with orange boxes, you can adjust it based on your project need as discussed below. Pay attention to the monthly price, make sure you claim only necessary HW resources, so that you can use your GCP instance for longer. Once you run out of credits, the VM instance will be shut down automatically and you might lose unsaved data and models. If you are almost running out of credits, contact the course staff. Change Configuration on Already Created VM Instances
Configure NetworkingWait until the deployment is finished. You should see a running VM with a green checkmark next to it on your Compute Engine page. We need to tweak a few more settings to enable remote access to Jupyter Notebook.
Your configuration sheets should look similar to below: Firewall Rules: Getting a Static IP AddressIf you want to have a static IP for ease of access, you can change the External IP address of your Google Cloud Engine instance to be static (see screenshot below). To Do this, click on the 3 line icon next to the Google Cloud Platform button on the top left corner of your screen, go to VPC network > External IP addresses (see screenshot below). To have a static IP address, change Type from Ephemeral to Static. Enter your prefered name for your static IP, ours is NOTE: At the end of CS 231N when you don't need your instance anymore, release the static IP address because Google charges a small fee for unused static IPs (according to this page). Take note of your Static IP address (circled on the screenshot below). We use 35.185.240.182 for this tutorial. Access Your Newly Created VMNow that you have created your virtual GCE, you want to be able to connect to it from your computer. The rest of this tutorial goes over how to do that using the command line. Install gcloud command-line ToolsTo access gcloud commands in your local terminal, install Google Cloud SDK that is appropriate for your platform and follow their instructions. If To ssh into your VM, go to your VM instance details page by clicking on its name. Start the VM instance first. Once it has a green check mark on, click on the drop-down arrow and select gcloud compute --project "<YOUR_PROJECT_ID>" ssh --zone "us-west1-b" "<YOUR_VM_NAME>" You should now be able to run Remote Server DevelopmentUsing Jupyter Notebook with Google Compute EngineIf you wish, you can use Jupyter Notebook to experiment in your projects. Below, we discuss how to run Jupyter Notebook from your GCE instance and connect to it with your local browser. After you SSH into the VM for the first time, you need to run a few commands in your home directory. You will be asked to set up a password for your Jupyter Notebook. git clone https://github.com/cs231n/gcloud.git
cd gcloud/
chmod +x ./setup.sh
./setup.sh Now you can run Jupyter Notebook from the folder with your assignment files.
The default port is You can connect to your Jupyter session from your personal laptop. Check the external ip address of your instance, say it is Verification
Transferring Files From Your Instance To Your ComputerFor instance, to transfer
For example, to download files from our instance to the current folder:
The transfer works in both directions. To upload a file to your instance:
If you would like to transfer an entire folder, you will need to add a resursive flag:
Other TipsYou can use Tmux to keep the training sessions running when you close your laptop. Also, if your collaborators log into the same account on the VM instance, they will see the same tmux session screen in real time. You can develop your code on remote server directly if you are comfortable with vim or emac. You can develop locally on your favorite editor, push to your branch on Github, and pull on remote server to run. (git commit frequently is also one of good Github practices) Besides BIG REMINDER: Make sure you stop your instances!Don't forget to stop your instance when you are done (by clicking on the stop button at the top of the page showing your instances). You can restart your instance and the downloaded software will still be available. We have seen students who left their instances running for many days and ran out of credits. You will be charged per hour when your instance is running. This includes code development time. We encourage you to read up on Google Cloud, regularly keep track of your credits and not solely rely on our tutorials. |
2023-10-27
2022-08-15
2022-08-17
2022-09-23
2022-08-13
请发表评论