mirror of
https://github.com/ToolJet/ToolJet
synced 2026-04-22 14:07:26 +00:00
108ac7bf64
* Revamp the documentation of airtable, amazonses and amazons3 * implement the changes based on the reviews * edited the changes to airtable, amazonses, s3 data sources in version 2.36.0 * addressed the reviews * align the images to the left
262 lines
No EOL
8.9 KiB
Markdown
262 lines
No EOL
8.9 KiB
Markdown
---
|
|
id: s3
|
|
title: Amazon S3
|
|
---
|
|
<div style={{paddingBottom:'24px'}}>
|
|
|
|
ToolJet can connect to **Amazon S3** buckets and perform various operation on them.
|
|
|
|
</div>
|
|
|
|
<div style={{paddingTop:'24px', paddingBottom:'24px'}}>
|
|
|
|
## Connection
|
|
|
|
To establish a connection with the Amazon S3 data source, you can either click on the **+Add new Data source** button located on the query panel or navigate to the **[Data Sources](/docs/data-sources/overview/)** page from the ToolJet dashboard.
|
|
|
|
ToolJet supports connecting to AWS S3 using **IAM Access Keys**, **AWS Instance Credentials** or **AWS ARN Role**.
|
|
|
|
If you are using **IAM Access Keys**, you will need to provide the following details:
|
|
|
|
- **Region**
|
|
- **Access key**
|
|
- **Secret key**
|
|
|
|
It is recommended to create a new IAM user for the database so that you can control the access levels of ToolJet.
|
|
|
|
<div style={{textAlign: 'left'}}>
|
|
<img style={{ border:'0', marginBottom:'15px' }} className="screenshot-full" src="/img/datasource-reference/aws-s3/iamnew.png" alt="aws s3 modal" />
|
|
</div>
|
|
|
|
To connect to AWS S3 using **AWS Instance Credentials**, select the **Use AWS Instance Credentials**. This will use the IAM role attached to the EC2 instance where ToolJet is running.
|
|
To access the metadata service of an ECS container and the EC2 instance, we use the WebIdentityToken parameter which is obtained from a successful login with an identity provider.
|
|
|
|
<div style={{textAlign: 'left'}}>
|
|
<img style={{ border:'0', marginBottom:'15px' }} className="screenshot-full" src="/img/datasource-reference/aws-s3/instanew.png" alt="aws s3 modal" />
|
|
</div>
|
|
|
|
If you are using **AWS ARN Role**, you will need to provide the following details:
|
|
|
|
- **Region**
|
|
- **Role ARN**
|
|
|
|
<div style={{textAlign: 'left'}}>
|
|
<img style={{ border:'0', marginBottom:'15px', width:'100%' }} className="screenshot-full" src="/img/datasource-reference/aws-s3/arnnew.png" alt="aws s3 modal" />
|
|
</div>
|
|
|
|
Click on **Test Connection** button to verify if the credentials are correct and that the database is accessible to ToolJet server. Click on **Save** button to save the data source.
|
|
|
|
:::tip
|
|
You can now connect to **[different S3 Hosts using custom endpoints](/docs/how-to/s3-custom-endpoints)**.
|
|
:::
|
|
|
|
</div>
|
|
|
|
<div style={{paddingTop:'24px', paddingBottom:'24px'}}>
|
|
|
|
## Querying AWS S3
|
|
|
|
Click on **+Add** button of the **[query manager](/docs/app-builder/query-panel/#add)** and select the data source added in the previous step as the data source. Select the operation that you want to perform, fill in the required parameters and click on **Run** button to run the query.
|
|
|
|
<div style={{textAlign: 'left'}}>
|
|
|
|
<img className="screenshot-full" src="/img/datasource-reference/aws-s3/operations3.png" alt="aws s3 query" />
|
|
|
|
</div>
|
|
|
|
<br/>
|
|
|
|
:::info
|
|
Query results can be transformed using transformations. Read our [transformations documentation](/docs/tutorial/transformations).
|
|
:::
|
|
|
|
</div>
|
|
|
|
<div style={{paddingTop:'24px', paddingBottom:'24px'}}>
|
|
|
|
## Query operations
|
|
|
|
You can create query for AWS S3 data source to perform several actions such as:
|
|
1. **[Create a new bucket](#create-a-new-bucket)**
|
|
2. **[Read object](#read-object)**
|
|
3. **[Upload object](#upload-object)**
|
|
4. **[Remove object](#remove-object)**
|
|
5. **[List buckets](#list-buckets)**
|
|
6. **[List objects in a bucket](#list-objects-in-a-bucket)**
|
|
7. **[Signed url for download](#signed-url-for-download)**
|
|
8. **[Signed url for upload](#signed-url-for-upload)**
|
|
|
|
</div>
|
|
|
|
<div style={{paddingTop:'24px', paddingBottom:'24px'}}>
|
|
|
|
### Create a new bucket
|
|
|
|
You can create a new bucket in your S3 by using this operation.
|
|
|
|
#### Required parameters:
|
|
|
|
- **Bucket Name**: Specify the bucket name.
|
|
|
|
<div style={{textAlign: 'left'}}>
|
|
|
|
<img className="screenshot-full" src="/img/datasource-reference/aws-s3/createbucket.png" alt="Create a new bucket - S3 operation" />
|
|
|
|
</div>
|
|
|
|
</div>
|
|
|
|
<div style={{paddingTop:'24px', paddingBottom:'24px'}}>
|
|
|
|
### Read object
|
|
|
|
You can read an object in a bucket by using this operation.
|
|
|
|
#### Required parameters:
|
|
|
|
- **Bucket**: Specify the bucket name.
|
|
- **Key**: Key of the object/file.
|
|
|
|
|
|
<div style={{textAlign: 'left'}}>
|
|
<img style={{ border:'0', marginBottom:'15px' }} className="screenshot-full" src="/img/datasource-reference/aws-s3/readv2-v3.png" alt="aws s3 read object" />
|
|
</div>
|
|
|
|
</div>
|
|
|
|
<div style={{paddingTop:'24px', paddingBottom:'24px'}}>
|
|
|
|
### Upload object
|
|
|
|
You can use this operation to upload objects(files) to your S3 bucket.
|
|
|
|
#### Required parameters:
|
|
|
|
- **Bucket**: Specify the bucket name.
|
|
- **Key**: Key of the object/file.
|
|
- **Content Type**: Specify file type such as text, image etc.
|
|
- **Upload data**: File/object that is to be uploaded.
|
|
|
|
|
|
<div style={{textAlign: 'left'}}>
|
|
<img style={{ border:'0', marginBottom:'15px' }} className="screenshot-full" src="/img/datasource-reference/aws-s3/uplobjv2-v3.png" alt="aws s3 upload"/>
|
|
</div>
|
|
|
|
</div>
|
|
|
|
<div style={{paddingTop:'24px', paddingBottom:'24px'}}>
|
|
|
|
### Remove object
|
|
|
|
You can use this operation to remove an object from your S3 bucket.
|
|
|
|
#### Required parameters:
|
|
|
|
- **Bucket**: Specify the bucket name.
|
|
- **Key**: Key of the object/file.
|
|
|
|
<div style={{textAlign: 'left'}}>
|
|
|
|
<img style={{ border:'0', marginBottom:'15px' }} className="screenshot-full" src="/img/datasource-reference/aws-s3/removeobject.png" alt="Create a new bucket - S3 operation" />
|
|
|
|
</div>
|
|
|
|
</div>
|
|
|
|
<div style={{paddingTop:'24px', paddingBottom:'24px'}}>
|
|
|
|
### List buckets
|
|
|
|
This operation will list all the buckets in your S3. This does not require any parameter.
|
|
|
|
|
|
<div style={{textAlign: 'left'}}>
|
|
<img style={{ border:'0', marginBottom:'15px' }} className="screenshot-full" src="/img/datasource-reference/aws-s3/listbucketsv2-v3.png" alt="aws s3 bucket" />
|
|
</div>
|
|
|
|
</div>
|
|
|
|
<div style={{paddingTop:'24px', paddingBottom:'24px'}}>
|
|
|
|
### List objects in a bucket
|
|
|
|
This operation will fetch the list of all the files in your bucket. It requires the following parameters:
|
|
|
|
#### Required parameters:
|
|
|
|
- **Bucket**: Specify the bucket name.
|
|
|
|
#### Optional parameters:
|
|
|
|
- **Prefix**: To limit the response to keys that begin with the specified prefix.
|
|
- **Max keys**: The maximum number of keys returned in the response body. Default value is 1000.
|
|
- **Offset**: The key to start with when listing objects in a bucket.
|
|
- **Next Continuation Token**: `Next Continuation Token` indicates Amazon S3 that the list is being continued on this bucket with a token. ContinuationToken is obfuscated and is not a real key.
|
|
|
|
|
|
:::info
|
|
**Next Continuation Token**
|
|
For listing a bucket for objects that begin with a specific character or a prefix, then use the `Offset` parameter. For example, if you want to list all the objects that begin with `a`, then set the `Offset` parameter to `a`. Similarly, if you want to list all the objects that begin with `ab`, then set the `Offset` parameter to `ab`.
|
|
|
|
The `Next Continuation Token` is used to list the next set of objects in a bucket. It is returned by the API when the response is truncated. The results will contain `Next Continuation Token` if there are more keys in the bucket that satisfy the list query. To get the next set of objects, set the `Next Continuation Token` parameter and run the query again.
|
|
The results will continue from where the last listing finished.
|
|
:::
|
|
|
|
|
|
<div style={{textAlign: 'left'}}>
|
|
|
|
<img style={{ border:'0', marginBottom:'15px' }} className="screenshot-full" src="/img/datasource-reference/aws-s3/listobjectsv2-v3.png" alt="aws s3 list object" />
|
|
|
|
</div>
|
|
|
|
</div>
|
|
|
|
<div style={{paddingTop:'24px', paddingBottom:'24px'}}>
|
|
|
|
### Signed url for download
|
|
|
|
The object owner can optionally share objects with others by creating a presigned URL, using their own security credentials, to grant time-limited permission to download the objects.
|
|
|
|
#### Required parameters:
|
|
|
|
- **Bucket**: Name of the bucket for uploading the file.
|
|
- **Key**: The object key.
|
|
- **Expires in**: The expiration time of URL.
|
|
|
|
|
|
<div style={{textAlign: 'left'}}>
|
|
|
|
<img style={{ border:'0', marginBottom:'15px' }} className="screenshot-full" src="/img/datasource-reference/aws-s3/urldownv2-v3.png" alt="aws s3 signed download" />
|
|
|
|
</div>
|
|
|
|
</div>
|
|
|
|
<div style={{paddingTop:'24px', paddingBottom:'24px'}}>
|
|
|
|
|
|
### Signed url for upload
|
|
|
|
The presigned URLs are useful if you want your user/customer to be able to upload a specific object to your bucket, but you don't require them to have AWS security credentials or permissions.
|
|
|
|
#### Required parameters:
|
|
|
|
- **Bucket**: Name of the bucket for uploading the file.
|
|
- **Key**: The object key.
|
|
- **Expires in**: The expiration time of URL.
|
|
- **Content Type**: The content type such as text, image etc.
|
|
|
|
|
|
<div style={{textAlign: 'left'}}>
|
|
|
|
<img style={{ border:'0', marginBottom:'15px' }} className="screenshot-full" src="/img/datasource-reference/aws-s3/urluplv2-v3.png" alt="aws s3 signed upload" />
|
|
|
|
</div>
|
|
|
|
|
|
|
|
:::info
|
|
We built an app to view and upload files to AWS S3 buckets. Check out the complete tutorial **[here](https://blog.tooljet.com/build-an-aws-s3-broswer-with-tooljet/)**.
|
|
:::
|
|
|
|
</div> |