These datasets are used in machine learning (ML) research and have been cited in peer-reviewed academic journals. Datasets are an integral part of the field of machine learning. Major advances in this field can result from advances in learning algorithms (such as deep learning), computer hardware, and, less intuitively, the availability of high-quality training datasets. High-quality labeled training datasets for supervised and semi-supervised machine-learning algorithms are usually difficult and expensive to produce because of the large amount of time needed to label the data. Although they do not need to be labeled, high-quality unlabeled datasets for unsupervised learning can also be difficult and costly to produce.

Many organizations, including governments, publish and share their datasets, often using common metadata formats (such as Croissant). The datasets are classified, based on the licenses, into two groups: open data and non-open data.

The datasets from various governmental-bodies are presented in List of open government data sites. The datasets are ported on open data portals. They are made available for searching, depositing and accessing through interfaces like Open API. The datasets are made available as various sorted types and subtypes.

List of sorting used for datasets TypeSubtypesSpecific categoryFinance, Economics, Commerce, Societal, Health, Academy, Sports, Food, Agriculture, Travel, Geospatial, Political, Consumer, Transport, Logistics, Environmental, Real-Estate, Legal, Entertainment, Energy, HospitalityScopeSupranational Union, National, Subnational, Municipality, Urban, RuralLanguageMandarin Chinese, Spanish, English, Arabic, Hindi, BengaliTypeTabular, Graph, Text, , Sound, VideoUsageTraining, validating, and testingFile-FormatsCSV, JSON, XML, KML, GeoJSON, Shapefile, GMLLicensesCreative-Commons, GPL, Other Non-Open data licensesLast-UpdatedLast-Hour, Last-Day, Last-Week, Last-Month, Last-YearFile-SizeMinimum, Maximum, RangeStatusVerified, In-Preparation, Deactivated(or Deprecated)Number of records100s, 1000s, 10000s, 100000s, MillionsNumber of variablesLess than 10, 10s, 100s, 1000s, 10000sServicesIndividual, Aggregation The data portal is classified based on its type of license. The open source license based data portals are known as open data portals which are used by many government organizations and academic institutions.

List of open data portals

Portal-nameLicenseList of installations of the portalTypical usagesComprehensive Knowledge Archive Network (CKAN)AGPLhttps://ckan.github.io/ckan-instances/

https://github.com/sebneu/ckan_instances/blob/master/instances.csvData repository for government or non-profit organisations, Data Management Solution for Research InstitutesDKANGPLhttps://getdkan.org/communityData repository for government or non-profit organisations, Data Management Solution for Research InstitutesDataverseApachehttps://dataverse.org/installations

https://dataverse.org/metricsData Management Solution for Research InstitutesDSpaceBSDhttps://registry.lyrasis.org/Data Management Solution for Research InstitutesOpenMLBSDhttps://www.openml.org/search?type=data&sort=runs&status=activeData Management Solution to share datasets, algorithms, and experiments results through APIs.

List of portals suitable for multiple types of applications

The data portal sometimes lists a wide variety of subtypes of datasets pertaining to many machine learning applications. Academic Torrentshttps://academictorrents.comAmazon Datasetshttps://registry.opendata.aws/Awesome Public Datasets Collectionhttps://github.com/awesomedata/awesome-public-datasetsdata.worldhttps://data.world/datasets/machine-learningDatahub – Core Datasetshttps://datahub.io/docs/core-dataDataONEhttps://www.dataone.org/DataPortalshttps://dataportals.org/Datasetlist.comhttps://www.datasetlist.comGlobal Open Data Index – Open Knowledge Foundation https://okfn.org/ Google Dataset Searchhttps://datasetsearch.research.google.com/Hugging Facehttps://huggingface.co/docs/datasets/IBM's Data Asset Exchangehttps://developer.ibm.com/exchanges/data/Jupyter – Tutorial Datahttps://jupyter-tutorial.readthedocs.io/en/latest/data-processing/opendata.htmlKagglehttps://www.kaggle.com/datasetsMachine learning datasetshttps://macgence.com/data-sets-and-cataloges/Major Smart Cities with Open Datahttps://rlist.io/l/major-smart-cities-with-open-data-portalsMicrosoft Datasetshttps://msropendata.com/datasetsOpen Data Inceptionhttps://opendatainception.io/Opendatasofthttps://data.opendatasoft.com/explore/dataset/open-data-sources%40public/table/?sort=code_enOpenDOARhttps://v2.sherpa.ac.uk/opendoar/OpenMLhttps://www.openml.org/search?type=dataPapers with Codehttps://paperswithcode.com/datasetsPenn Machine Learning Benchmarkshttps://github.com/EpistasisLab/pmlb/tree/master/datasetsPublic APIshttps://github.com/public-apis/public-apisRegistry of Open Access Repositorieshttp://roar.eprints.org/  REgistry of REsearch Data REpositorieshttps://www.re3data.org/  UCI Machine Learning Repositoryhttps://archive.ics.uci.edu/Speech Datasethttps://www.shaip.com/offerings/speech-data-catalog/Visual Data Discoveryhttps://visualdata.io/discovery

List of portals suitable for a specific subtype of applications

The data portals which are suitable for a specific subtype of machine learning application are listed in the subsequent sections.

Image data

Text data These datasets consist primarily of text for tasks such as natural language processing, sentiment analysis, translation, and cluster analysis.

Reviews Dataset name Brief description Preprocessing Instances Format Default task Created (updated) Reference CreatorNetflix PrizeMovie ratings on Netflix.100,480,507 ratings that 480,189 users gave to 17,770 moviesText, ratingRating prediction2006NetflixAmazon reviews US product reviews from Amazon.com.None. 233.1 millionTextClassification, sentiment analysis2015 (2018)McAuley et al.OpinRank Review DatasetReviews of cars and hotels from Edmunds.com and TripAdvisor respectively.None.42,230 / ~259,000 respectivelyTextSentiment analysis, clustering2011K. Ganesan et al.MovieLens22,000,000 ratings and 580,000 tags applied to 33,000 movies by 240,000 users.None.~ 22MTextRegression, clustering, classification2016GroupLens ResearchYahoo! Music User Ratings of Musical ArtistsOver 10M ratings of artists by Yahoo users.None described.~ 10MTextClustering, regression2004Yahoo!Car Evaluation Data SetCar properties and their overall acceptability.Six categorical features given.1728TextClassification1997Bohanec, Marko, and Vladislav Rajkovic. "Knowledge acquisition and explanation for multi-attribute decision making." 8th Intl Workshop on Expert Systems and their Applications. 1988.Tan, Peter J., and David L. Dowe. "MML inference of decision graphs with multi-way joins." Australian Joint Conference on Artificial Intelligence. 2002.M. BohanecYouTube Comedy Slam Preference DatasetUser vote data for pairs of videos shown on YouTube. Users voted on funnier videos.Video metadata given.1,138,562TextClassification2012GoogleSkytrax User Reviews DatasetUser reviews of airlines, airports, seats, and lounges from Skytrax.Ratings are fine-grain and include many aspects of airport experience.41396TextClassification, regression2015Q. NguyenTeaching Assistant Evaluation DatasetTeaching assistant reviews.Features of each instance such as class, class size, and instructor are given.151TextClassification1997Loh, Wei-Yin, and Yu-Shan Shih. "Split selection methods for classification trees." Statistica sinica(1997): 815–840.W. Loh et al.Vietnamese Students' Feedback Corpus (UIT-VSFC)Students' Feedback.Comments16,000TextClassification1997Nguyen et al.Vietnamese Social Media Emotion Corpus (UIT-VSMEC)Users' Facebook Comments.Comments6,927TextClassification1997Nguyen et al.Vietnamese Open-domain Complaint Detection dataset (ViOCD)Customer product reviewsComments5,485TextClassification2021Nguyen et al.ViHOS: Hate Speech Spans Detection for VietnameseSocial Media TextsCommentsContaining 26k spans on 11k commentsTextSpan Detection2021Hoang et al.

News articles Dataset name Brief description Preprocessing Instances Format Default task Created (updated) Reference CreatorNYSK DatasetEnglish news articles about the case relating to allegations of sexual assault against the former IMF director Dominique Strauss-Kahn.Filtered and presented in XML format.10,421XML, textSentiment analysis, topic extraction2013Dermouche, M. et al.The Reuters Corpus Volume 1Large corpus of Reuters news stories in English.Fine-grain categorization and topic codes.810,000TextClassification, clustering, summarization2002ReutersThe Reuters Corpus Volume 2Large corpus of Reuters news stories in multiple languages.Fine-grain categorization and topic codes.487,000TextClassification, clustering, summarization2005ReutersThomson Reuters Text Research CollectionLarge corpus of news stories.Details not described.1,800,370TextClassification, clustering, summarization2009T. Rose et al.Saudi Newspapers Corpus31,030 Arabic newspaper articles.Metadata extracted.31,030JSONSummarization, clustering2015M. AlhagriRE3D (Relationship and Entity Extraction Evaluation Dataset)Entity and Relation marked data from various news and government sources. Sponsored by DstlFiltered, categorisation using Baleen typesnot knownJSONClassification, Entity and Relation recognition2017DstlExaminer Spam Clickbait CatalogueClickbait, spam, crowd-sourced headlines from 2010 to 2015Publish date and headlines3,089,781CSVClustering, Events, Sentiment2016R. KulkarniABC Australia News CorpusEntire news corpus of ABC Australia from 2003 to 2019Publish date and headlines1,186,018CSVClustering, Events, Sentiment2020R. KulkarniWorldwide News – Aggregate of 20K FeedsOne week snapshot of all online headlines in 20+ languagesPublish time, URL and headlines1,398,431CSVClustering, Events, Language Detection2018R. KulkarniReuters News Wire Headline11 Years of timestamped events published on the news-wirePublish time, Headline Text16,121,310CSVNLP, Computational Linguistics, Events2018R. KulkarniThe Irish Times Ireland News Corpus24 Years of Ireland News from 1996 to 2019Publish time, Headline Category and Text1,484,340CSVNLP, Computational Linguistics, Events2020R. KulkarniNews Headlines Dataset for Sarcasm DetectionHigh quality dataset with Sarcastic and Non-sarcastic news headlines.Clean, normalized text26,709JSONNLP, Classification, Linguistics2018Rishabh Misra

Messages