New AI Dataset Based on LibGen and Sci-Hub

Hacker News

We recently began extracting the text layers of scholarly publications and books to include in our database. This encompasses sources such as scimag, libgen, and the latest zlib leaks. Our project, named the Standard Template Construct, also features a distributed search engine and incorporates various AI routines to handle the text corpus. Today we have releases our first dataset, STC230908. This dataset contains approximately 75,000 book texts, 1.3 million scholarly paper texts, and 24 million abstracts, including the years from 2021 to 2023. We're currently in the process of preparing the next version of the dataset, which will include an additional 300,000 books. How to Access Short Instructions: Install IPFS and launch it. pip3 install stc-geck && geck - documents More details: the dataset is released in IPFS and replicated to multiple nodes. It is in format of database for the search engine that we use in STC. GECK is the library that embeds this search engine and allows to stream all contained data in easy way. Even more detailed Instructions: https://github.com/nexus-stc/stc/tree/master/geck

Visit product View on Hacker NewsSep 8, 2023

Share card

Actual performance

4points

Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →

Hacker NewsStrong engagement from HN community · Strong signals: 000, io, including · Missing: https docs, excited, just released

85%85% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

Indie HackersFits the IH revenue-focused audience · Strong signals: including · Missing: supports, reddit linkedin, podcasting

80%80% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

AppSumoStrong fit for a featured deal · Missing: plus, platform, intuitive

53%53% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Product HuntUnlikely to reach the leaderboard · Strong signals: new · Missing: mac, agents, macos

49%49% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

TrustMRRLess likely to generate early MRR · Strong signals: way · Missing: mobile apps, ios, personal

31%31% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

19%19% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Incorrect prediction on native model

Similar products

I made a new AI colorizer57%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

I made a new AI colorizer

Hacker News472

New AI edits images based on text instructions65%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

New AI edits images based on text instructions

Hacker News1,098

New AI Grant Fellows63%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

New AI Grant Fellows

Hacker News9

Mynd, a new AI journaling app17%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Mynd, a new AI journaling app

Hacker News3

My new OpenVZ-oriented distro based on el663%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

My new OpenVZ-oriented distro based on el6

Hacker News4

HarmonyCAD54%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

A New AI CAD tool

Indie Hackerscommitment-side-project

AlgoEdit44%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

New AI assistant for editing

Indie Hackerscommitment-full-time

Infra.new, an AI Agent for Terraform41%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Infra.new, an AI Agent for Terraform

Hacker News2

A new AI Based tool that will help developers understand new codebase33%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

A new AI Based tool that will help developers understand new codebase

Hacker News2

CrasAI35%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Your all-in-one ai hub

Product Hunt

Browse all products Analyze your launch