Google Cloud Storage is a scalable, fully-managed object storage service provided by Google Cloud Platform (GCP). It allows you to store and retrieve any amount of data securely, making it a reasonable choice for storing various types of files, such as images, videos, backups, and more. With Google Cloud Storage, you can create buckets (containers) to organize and manage your objects (files) and define access control and permissions for different users.
To upload a file to Google Cloud Storage using Python 3, you'll need to perform the following steps:
Step 1: Create a Google Cloud Platform (GCP) project:
Go to the GCP Console
Create a new project or use an existing one.
Step 2: Create a new service account:
Go back to the Google Cloud Console.
In the left navigation menu, go to "IAM & Admin" > "Service accounts".
Click on the "Create Service Account" button.
Step 3: Fill in the Service Account details:
Enter a name for your service account.
You can also add a description (optional).
Step 4: Assign a role to the service account:
Choose the appropriate role for your use case.
For GCS access, the "Storage Object Admin" role provides sufficient permissions.
If you need more fine-grained control, you can choose the necessary roles accordingly.
Step 5: Create and download the JSON key file:
Click on the "Create key" button.
Select the JSON key type.
Click on the "Create" button.
The JSON file containing your service account credentials will be downloaded to your system. Make sure to keep this file safe and do not share it publicly.
Step 6: Create the Bucket on Google Cloud Storage
Navigate to "Google Cloud Storage"
On the left-hand side navigation pane, click on "Storage" > "Storage" to access Google Cloud Storage.
In the Google Cloud Storage section, click the "Create Bucket" button to begin the process of creating a new bucket.
Provide Bucket details. You will now be taken to the "Create a bucket" page.
Click "Create" after providing the required details, click the "Create" button to create the bucket.
Step 7: Install the required Python library:
You need to install the Google Cloud Storage library for Python, which provides the necessary functionalities to interact with Google Cloud Storage.
Use pip to install it:
pip install google-cloud-storage
Step 8: Write Python code to upload a file to Google Cloud Storage:
Now, let's write a Python script to upload a file to Google Cloud Storage using the credentials.json
file.
from google.cloud import storagedef upload_to_gcs(bucket_name, source_file_path, destination_blob_name, credentials_file):# Initialize the Google Cloud Storage client with the credentialsstorage_client = storage.Client.from_service_account_json(credentials_file)# Get the target bucketbucket = storage_client.bucket(bucket_name)# Upload the file to the bucketblob = bucket.blob(destination_blob_name)blob.upload_from_filename(source_file_path)print(f"File {source_file_path} uploaded to gs://{bucket_name}/{destination_blob_name}")if __name__ == "__main__":# Replace the following variables with your specific valuesBUCKET_NAME = "your-bucket-name"SOURCE_FILE_PATH = "path/to/your/local/file.txt"DESTINATION_BLOB_NAME = "uploaded-file.txt"CREDENTIALS_FILE = "path/to/your/credentials.json"upload_to_gcs(BUCKET_NAME, SOURCE_FILE_PATH, DESTINATION_BLOB_NAME, CREDENTIALS_FILE)
Note:
BUCKET_NAME
: Replaceyour-bucket-name
with the name of the bucket you created on Google Cloud Storage. The bucket name must be globally unique across all of Google Cloud Storage, so choose a name that reflects the purpose of the bucket and is not likely to be used by others.
SOURCE_FILE_PATH
: Replacepath/to/your/local/file.txt
with the local file path of the file you want to upload to Google Cloud Storage. This should be the absolute or relative path to the file on your local machine that you want to store in the cloud.
DESTINATION_BLOB_NAME
: Replaceuploaded-file.txt
with the desired name of the file that will be shown in the bucket after the upload. When you upload the file, it will be stored in the bucket with this specified name.
CREDENTIALS_FILE
: Replacepath/to/your/credentials.json
with the file path of the JSON credentials file you downloaded when creating the service account. This JSON file contains the necessary authentication information for the service account, allowing your Python script to interact with Google Cloud Storage securely.
Line 1: Here, we import the storage
module from the google.cloud
package. This module provides the necessary functionality to interact with Google Cloud Storage.
Line 3–12: In this section, we define a function named upload_to_gcs
that takes four parameters: bucket_name
, source_file_path
, destination_blob_name
, and credentials_file
. This function will be responsible for uploading a file to Google Cloud Storage.
Line 5: In this part, we create a new storage.Client
instance called storage_client
by using the from_service_account_json
method provided by the Client
class. We pass the credentials_file
parameter to this method to authenticate the client with the provided service account credentials.
Line 8: Using the initialized storage_client
, we retrieve the specified bucket by calling the bucket
method with the bucket_name
parameter. This step allows us to interact with the bucket and its objects.
Line 10–12: In this section, we create a new blob (object) within the target bucket with the specified destination_blob_name
. The blob.upload_from_filename
method is used to upload the file from the local source_file_path
to the blob in the bucket.
Line 14: After the file is successfully uploaded, we print a message to indicate the success and display the URL of the uploaded object in the Google Cloud Storage bucket.
Line 16–23: In this part, we have the main code execution block. We define the specific values for BUCKET_NAME
, SOURCE_FILE_PATH
, DESTINATION_BLOB_NAME
, and CREDENTIALS_FILE
, and then call the upload_to_gcs
function to upload the file to Google Cloud Storage using the provided credentials and configurations.
Free Resources