cloudquery - GitLite

KO | EN

gitlite — search

#javascript #python #react #typescript #go #android #deep-learning #docker #awesome #awesome-list #vue #pytorch #security #openai

cloudquery

(★ 2,150)

Turn any website to API by several clicks (serverless and support SPA!)

File Explorer

Download Latest Version (.zip)

- FUNDING.yml
- config.js
- index.js
- favicon.ico
- index.html
- index.jsx
.editorconfig
.eslintignore
.eslintrc
.gitignore
.gitpod.yml
.upignore
LICENSE
package-lock.json
package.json
README.md
up.json

🚀 Installation Guide

1. Get the code

git clone https://github.com/t9tio/cloudquery

Downloads the entire project code from GitHub to your computer.

cd cloudquery

Moves into the project folder you just downloaded.

2. Node.js

Easy Recommended

Prerequisites

Git Needed to download the project code from GitHub.
Node.js Node.js must be installed to use npm. The LTS version is recommended.

npm i

Type this command into your terminal and run it.

npm run sb

Type this command into your terminal and run it.

npm run sf

Type this command into your terminal and run it.

✅ After running the command, open the address shown in the terminal (usually something like http://localhost:3000) in your browser.

Pulled directly from this repo's README.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# CloudQuery

[![Join us](https://badgen.net/badge/Join%20the%20community%20of%20t9t.io/Get%20in%20touch/green)](https://t9t.io/#contact)

Turn any website to API by several clicks.

Try online: https://cloudquery.t9t.io

![](https://user-images.githubusercontent.com/5512552/51655619-6b978580-1fd8-11e9-93a9-9accf8e25e54.gif)

## API

- Sample Query: [Best 3 posts on HN](https://cloudquery.t9t.io/query?url=https%3A%2F%2Fnews.ycombinator.com%2F&selectors=*:nth-child(3)%20*:nth-child(1)%20%3E%20*:nth-child(3)%20%3E%20*:nth-child(1),*:nth-child(4)%20%3E%20*:nth-child(3)%20%3E%20*:nth-child(1),*:nth-child(7)%20%3E%20*:nth-child(3)%20%3E%20*:nth-child(1))

Query params:

- url: url of the webpage you want to fetch
- selectors: css selectors describing the elements interests you

Returns:

- innerText: element innerText
- href: element href

## Run locally

```bash
# install dependencies
npm i
# start backend server(for auto restart when code change)
npm run sb
# start frontend server(for auto restart when code change)
npm run sf

# visit http://localhost:1234
```

## How to deploy your own CloudQuery to AWS lambda

### 1. Configuration

- update `profile`([AWS Credential Profiles](https://up.docs.apex.sh/#aws_credentials)) in `up.json` to use your own aws account
- update `rateLimit` in `config.json` to set your own rate limit (Default: 5 request/hour)

### 2. Deploy

```bash
# deploy to aws lambda
up

# see the URL of your CloudQuery
up url

# deploy production version(to cloudquery.t9t.io)
up deploy production

```

# Thanks

- [up](https://github.com/apex/up) for deploying serverless API to AWS with ease
- [serverless-chrome](https://github.com/adieuadieu/serverless-chrome) for running chrome on AWS lambda
- [finder](https://github.com/antonmedv/finder) for making it easy to select elements on webpage