How to create a universal Telegram bot – parser of any posts from channels with AI-normalization and fallback rules

mkdir tg-universal-parser-bot
cd tg-universal-parser-bot
npm init -y
npm install telegraf node-cron axios cheerio groq-sdk dotenv
npm install -D nodemon

BOT_TOKEN=твой_токен_из_BotFather
GROQ_API_KEY=твой_groq_ключ
SUPABASE_URL=https://...
SUPABASE_ANON_KEY=eyJ...

Create a full working bot framework on Telegraf v4.

Requirements:
- Use dotenv.
Commands: /start, /addsource, /listsources, /status
Inline buttons and single-navigation (update one message)
Prepare a separate worker.js file for the cronworker (every 10 minutes)
Prepare folders: src/bot, src/worker, src/parsers, src/ai, src/db
Connect Supabase (or SQLite as Fallback)
It should be run by npm start and npm run worker.
- Add comments "// TODO" where you will need to insert parser logic and AI

Output ALL project code as a structure of content files.

Based on the previous code, add a complete universal parser architecture for any posts from Telegram channels.

Supabase tables (SQL migration):
- sources (id, channel username, type: public|private, cursor, last import)
raw posts (id, source id, message id, raw text, posted at)
- parsed posts (id, raw id, post type, category, title, summary, links: jsonb, contacts: jsonb, confidence, status: published|skipped, published at)

Logic:
Every 10 minutes a worker visits sources.
- for public: parsite https://t.me/s/...
- for private: stub (later add MTProto)
- After receiving raw → AI parse → fallback rules → auto decision (publish if confidence > 75 and there is useful content)
- No manual moderation! Only auto-publish or skip + cause log

Update files: db/migrations.sql, src/ai/parser.js, src/worker/importer. js

Based on the previous code, implement a full-fledged parser of public channels.

ParsePublicChannel(channel: string, afterTimestamp?: number):
- axios.get (`https://t.me/s/${channel}`)
- cheerio parses all posts
- Retrieves: message id, full text, date, all links, all @username, buttons
- Removes noise: channel footers, advertising, "subscribe", repeated blocks
Returns an array of clean raw posts

Add an anti-duplicate to (channel + message id)

Create two Prompts for Groq (llama-3.3-70b or mixtral):

1. The main AI prompt (returns only JSON):
Systemic: You are an expert in structuring Telegram posts. Return only valid JSON without markdown.

JSON diagram:
{
“post type: “announcement | sale | question | news | offer | discussion | other”,
Category: String (one word or short phrase)
"title": "string,"
"summary": "stringing up to 300 characters"
"links": ["array of urls"],
“contacts: [”@username”, “phone”, “other”],
"confidence": 0-100,
"Reason": "stringing short"
}

2. Fallback rules (if AI is not available or confidence < 60):
- If you have @username or a link to an external resource → confidence 70
If more than 100 characters of meaningful text → confidence 50
Extract all links and @ via regex

Implement the function normalizePost(rawText) which first tries AI, if the error - fallback.

Add the decision engine without manual moderation:

Rules:
- If confidence >= 78 AND (are contacts OR links.length > 0) → status: "published"
- Otherwise, "skipped" with the preservation of the cause in the database
- For published - save in parsed posts and (optional) send to the channel bot or subscriber's face n

Implement decideAndSave(parsedData)

Add:
Simple tests (jest or console.log) for parser and AI
- Team in package.json: "dev": "nodemon src/bot/index.js"
- “worker”: “node src/worker/importer.js”
In /start show a beautiful keyboard with buttons

Get the final launch instructions.

npm run dev

npm run worker