SharePointScavenger

KO | EN

gitlite — search

#hacktoberfest #ai #typescript #machine-learning #rust #docker #nodejs #windows #awesome #cli #kubernetes #vue #awesome-list #open-source

SharePointScavenger

(★ 10)

SharePointScavenger is an automated SharePoint scraper used to find credentials and secrets.

SharePointScavenger 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# SharePointScavenger

```
   _____ __                    ____        _       __ 
  / ___// /_  ____ _________  / __ \____  (_)___  / /_
  \__ \/ __ \/ __ `/ ___/ _ \/ /_/ / __ \/ / __ \/ __/
 ___/ / / / / /_/ / /  /  __/ ____/ /_/ / / / / / /_  
/____/_/ /_/\__,_/_/   \___/_/    \____/_/_/ /_/\__/  
  / ___/_________ __   _____  ____  ____ ____  _____  
  \__ \/ ___/ __ `/ | / / _ \/ __ \/ __ `/ _ \/ ___/  
 ___/ / /__/ /_/ /| |/ /  __/ / / / /_/ /  __/ /      
/____/\___/\__,_/ |___/\___/_/ /_/\__, /\___/_/       
                                 /____/

v0.1
    mwrcybersec.com

```

SharePointScavenger is a python script to scrape SharePoint for secrets and sensitive files. Search terms and queries are specified in a file that can be modified for ogranisation specific secrets. It supports both a searching based on predefined queries and a downloading based on previous search results.

How it works:
* Microsoft's MSAL library is used for authentication using the device auth code flow 
* [Roadtx](https://dirkjanm.io/introducing-roadtools-token-exchange-roadtx/) is used for interactive authentication. Useful if device code auth flow is blocked by conditional access.
* Mix of Sharepoint APIs and Microsoft Graph API are used to search for potentially sensitive files defined in a queries list file. If a server error is encountered, the query is placed back on the list to be re-executed. 404's are not re-executed because this assumed the indexed file has been removed
* Access tokens can be saved to disk for re-use. If the token expires within 10 minutes, the refresh token will be used to request new tokens.

## Requirements

Folder structure is already generated in the Git project, but the following folders should be present:
* `files/` - This is where files downloaded from the download mode are saved
* `log/` - Self-explanatory, but this is where logs are stored from execution
* `results/` - Folder where CSV files with the results generated from scraping are stored

Python Dependencies are listed in requirements.txt:
```
pip install -r ./requirements.txt
```

## General Usage

The script currently supports two modes (`search` and `download`), which do the following:
- **Search**: This mode implements the core function of scraping SharePoint for search terms defined in `queries.txt`. Results are stored in a CSV format, all attributes defined in `file_attributes` array in `SharePointScavenger.py`, are logged to the CSV.
- **Download**: This mode is used to download files based off a CSV of previous results. This can be used for offline processing of these files.

```
python3 ./SharePointScavenger.py [mode] [args]
```

## Search Usage

This mode will output two CSV files, `results/<date_time>_output.csv` and `results/<date_time>_output_unique.csv`. The latter is an attempt at reducing file duplication when multiple search terms return the same file, this should reduce the overhead for the download function.

```
$ python3 ./SharePointScavenger.py search --help

_____ __                    ____        _       __ 
  / ___// /_  ____ _________  / __ \____  (_)___  / /_
  \__ \/ __ \/ __ `/ ___/ _ \/ /_/ / __ \/ / __ \/ __/
 ___/ / / / / /_/ / /  /  __/ ____/ /_/ / / / / / /_  
/____/_/ /_/\__,_/_/   \___/_/    \____/_/_/ /_/\__/  
  / ___/_________ __   _____  ____  ____ ____  _____  
  \__ \/ ___/ __ `/ | / / _ \/ __ \/ __ `/ _ \/ ___/  
 ___/ / /__/ /_/ /| |/ /  __/ / / / /_/ /  __/ /      
/____/\___/\__,_/ |___/\___/_/ /_/\__, /\___/_/       
                                 /____/

v0.1
    mwrcybersec.com

usage: SharePointScavenger.py mode [options]

positional arguments:
  {search,download}     Mode to run the script. Search for files or download from a list

options:
  -h, --help            show this help message and exit
  --authmethod {devicecode,interactive}
                        Authentication method to use, defaults to devicecode. If this is blocked by conditional access you can use an interactive login (shout out to @dirkjanm - https://github.com/dirkjanm/ROADtools)
  -u USERNAME, --username USERNAME
                        Username for authentication
  -t TOKEN, --token TOKEN
                        Provide an access token to use
  -p PROXY, --proxy PROXY
                        Specify an HTTP proxy to use. NB! Sets SSL verify to false!
  --use-previous-token  Use a previous access token stored in previous_token.json
  --threads THREADS     Number of threads to use
  -U URL, --url URL     Site URL for SharePoint
  -q QUERIES_FILE, --queries-file QUERIES_FILE
                        Input file of search queries to run
  --row-limit ROW_LIMIT
                        Row Limit for queries, Default = 50000

```

**Required Arguments:**
* -u/--username
* -U/--url

**Example Usage:**
```
$ python3 ./SharePointScavenger.py search \
    --url https://mydomain.sharepoint.com/ \
    --username 'alice@mydomain.com' \
    --row-limit 50000 \
    --queries-file ./queries.txt  \
    --use-previous-token
```

## Download Usage
* You will need a valid results file from `search` to run this mode
* By default a prompt will be given for files over 50mb, use `--no-prompt` to override, but do so at your own risk, especially if you happen to find virtual hard disks
* Files with duplicate names are suffixed with the collision number, the log file `log/<date_time>.csv` will save the original URL and downloaded filename for cross correlating.

```
$ python3 ./SharePointScavenger.py download --help

_____ __                    ____        _       __ 
  / ___// /_  ____ _________  / __ \____  (_)___  / /_
  \__ \/ __ \/ __ `/ ___/ _ \/ /_/ / __ \/ / __ \/ __/
 ___/ / / / / /_/ / /  /  __/ ____/ /_/ / / / / / /_  
/____/_/ /_/\__,_/_/   \___/_/    \____/_/_/ /_/\__/  
  / ___/_________ __   _____  ____  ____ ____  _____  
  \__ \/ ___/ __ `/ | / / _ \/ __ \/ __ `/ _ \/ ___/  
 ___/ / /__/ /_/ /| |/ /  __/ / / / /_/ /  __/ /      
/____/\___/\__,_/ |___/\___/_/ /_/\__, /\___/_/       
                                 /____/

v0.1
    mwrcybersec.com

usage: SharePointScavenger.py mode [options]

positional arguments:
  {search,download}     Mode to run the script. Search for files or download from a list

options:
  -h, --help            show this help message and exit
  --authmethod {devicecode,interactive}
                        Authentication method to use, defaults to devicecode. If this is blocked by conditional access you can use an interactive login (shout out to @dirkjanm - https://github.com/dirkjanm/ROADtools)
  -u USERNAME, --username USERNAME
                        Username for authentication
  -t TOKEN, --token TOKEN
                        Provide an access token to use
  -p PROXY, --proxy PROXY
                        Specify an HTTP proxy to use. NB! Sets SSL verify to false!
  --use-previous-token  Use a previous access token stored in previous_token.json
  --threads THREADS     Number of threads to use
  -U URL, --url URL     Site URL for Sharepoint
  -i INPUT_FILE, --input-file INPUT_FILE
                        Input file with list of files to download
  --no-prompt           Do not prompt for files over 50mb. NB! Be careful with disk space.
  --delete-old-files    Clears previously downloaded files in sharepoint_search.file_downloadPath [/files by default]. THIS WILL DELETE FILES!

```

**Required Arguments:**
* -u/--username
* -U/--url
* -i/--input-file

**Example Usage:**
```
python3 ./SharePointScavenger.py  download  \
    -i results/2025-04-30_09-24-35_output_unique.csv   \
    -u 'alice@mydomain.com'  \
    --url https://mydomain.sharepoint.com/ \
    --use-previous-token \
    --delete-old-files \
    --threads 15
```

## Acknowledgements

@dirkjanm for the epic ROADtools toolset, the interactive authentication flow uses roadtx to get the authentication tokens: 
* https://dirkjanm.io/introducing-roadtools-token-exchange-roadtx/

The device code auth flow and output formatting was taken from TeamsEnum:
* https://github.com/sse-secure-systems/TeamsEnum

SharePointScavenger

SharePointScavenger 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

SharePointScavenger

SharePointScavenger 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README