PostyAI

A LinkedIn post generator that works, and a post scorer that fell short.

Context

It started in December 2024 as a way to post on LinkedIn consistently, in your own voice. Six months later I rebuilt it in Flask around a harder question: can a model tell you whether a post will land before you publish it? I built it alone, for fun, and the answer is the interesting part.

Four inputs, and no prompt box

Topic, length, language and tone fill one fixed prompt, so every draft comes out in the same shape. That was for the scorer: a model can’t compare posts whose shape changes on every run.

Nobody writes one post

Content work arrives as a campaign, so batch generation writes two to five posts against the same settings. You pick from a set instead of regenerating until something lands.

Seventy-three posts can’t predict engagement

I built about 85 features per post, from readability and sentiment to the hook and the density of emoji and hashtags, and trained nine regressors on 73 posts. The best, a random forest, explained 65% of the variance on about fifteen held-out posts. That is too few posts to trust any of the nine.

73

Posts to learn from

65%

Of the variance explained by the best of nine models

~15

Posts it was tested on

The score on screen is a rule

The model’s features never matched the live pipeline, so the app never served it. The 8/10 under a draft is a rule I wrote: points for length, a few emoji and some formatting. It checks the shape of a post, not its engagement.

What I would fix next

More data before more models. Nine regressors had nothing to learn from 73 posts; the next version collects outcomes from posts people actually publish, and only then earns a score, shown with the reasons behind it.

Next case study

A forgery detector that shows its work