Scalable-Hashtag-Recommender-System

KO | EN

gitlite — search

#javascript #typescript #llm #golang #go #machine-learning #java #nodejs #windows #awesome #kubernetes #awesome-list #vue #security

Scalable-Hashtag-Recommender-System

(★ 12)

Scalable hashtag recommender system based on mini-batch fast k-means and neural networks

Scalable-Hashtag-Recommender-System 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Scalable Hashtag Recommender System

A hashtag recommender system based on k-means, mini-batch fast k-means and a deep learning feature extraction phase.[1]
  Usage: hashtag-recommender-system [OPTIONS]
  Recommends appropriate hashtags for a given image.

Check OPTIONS.md for the list of the possible parameters.

# Deploy on Amazon aws via Flintrock

- Run deploy.sh inside the "deploy" folder. The script is configured to upload another script on the master and on each slave machine. It copies the jar file on each machine. Finally it runs the previously uploaded script "nodesetup" on each machine in parallel. This script downloads and installs all the dependencies (e.g. python, python libraries ecc.)
N.B in order to let flintrock access aws machine you need to export the aws keys and id as system keys via:
export AWS_ACCESS_KEY_ID="yourawsid"
export AWS_SECRET_ACCESS_KEY="yourawskey"

- Conf.yaml is used by flintrock to configure the cluster properties. One can choose the OS, number of slaves ecc.

- Once installed all the dependencies, in order to run the jar one needs to login via ssh to the master slave. Then run a script like this:

spark-submit --master spark://ip-<masterip> --driver-memory 25G --executors-memory 28G 
--executor-core 8 --class "Main" --deploy-mode client  shrs.jar \
 -f file_path.csv -t tag_listc.txt --image-path "https://image.ibb.co/kYdbKT/IMG_20180725_194058_490.jpg" 
 -c minibatch -i 20 -b 10000  -m spark://ip-<masterip>:7077
  
  driver-memory, executors-memory and executor core are parameters of master and slaves machines. Spark default uses only 1 GB on each machine.

[1]=https://www.eecs.tufts.edu/~dsculley/papers/fastkmeans.pdf

Scalable-Hashtag-Recommender-System

Scalable-Hashtag-Recommender-System 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Scalable-Hashtag-Recommender-System

Scalable-Hashtag-Recommender-System 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README