DravidianCodeMix-Dataset

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

DravidianCodeMix-Dataset

★ 20 Open GitHub ↗

No description available.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# DravidianCodeMix-Dataset

DravidianCodeMix: Sentiment Analysis and Offensive Language Identification Dataset for Dravidian Languages in Code-Mixed Text

## Full dataset is in zip file named "DravidianCodeMix-2020.zip"

This paper describes the development of a multilingual, manually annotated dataset for three under-resourced Dravidian languages generated from social media comments. The dataset was annotated for sentiment analysis and offensive language identification for a total of more than 60,000 YouTube comments. The dataset consists of around 44,000 comments in Tamil-English, around 7,000 comments in Kannada-English, and around 20,000 comments in Malayalam-English. The data was manually annotated by volunteer annotators and has a high inter-annotator agreement in Krippendorff's alpha. The dataset contains all types of code-mixing phenomena since it comprises user-generated content from a multilingual country.  We also present baseline experiments to establish benchmarks on the dataset using machine learning methods.

If you are using the data or code from this research then please site our paper below:

@article{Chakravarthi2022,
    author={Chakravarthi, Bharathi Raja
    and Priyadharshini, Ruba
    and Muralidaran, Vigneshwaran
    and Jose, Navya
    and Suryawanshi, Shardul
    and Sherly, Elizabeth
    and McCrae, John P.},
    title={DravidianCodeMix: sentiment analysis and offensive language identification dataset for Dravidian languages in code-mixed text},
    journal={Language Resources and Evaluation},
    year={2022},
    month={Feb},
    day={04},
    abstract={This paper describes the development of a multilingual, manually annotated dataset for three under-resourced Dravidian languages generated from social media comments. The dataset was annotated for sentiment analysis and offensive language identification for a total of more than 60,000 YouTube comments. The dataset consists of around 44,000 comments in Tamil-English, around 7000 comments in Kannada-English, and around 20,000 comments in Malayalam-English. The data was manually annotated by volunteer annotators and has a high inter-annotator agreement in Krippendorff's alpha. The dataset contains all types of code-mixing phenomena since it comprises user-generated content from a multilingual country. We also present baseline experiments to establish benchmarks on the dataset using machine learning and deep learning methods. The dataset is available on Github and Zenodo.},
    issn={1574-0218},
    doi={10.1007/s10579-022-09583-7},
    url={https://doi.org/10.1007/s10579-022-09583-7}
    }

Data can be accessed by downloading zip file or https://zenodo.org/record/4750858#.YJtw0SYo_0M or mailing author bharathiraja.akr@gmail.com

DravidianCodeMix-Dataset

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README