fix facebook feed decoy junk extraction - #3
Merged
lawmight merged 1 commit intoSep 10, 2026
Merged
Conversation
…tent Filter Adolph-style decoy rows where Facebook injects plausible author names but scrambled alphanumeric soup for post bodies. Adds in-page isScrambledOrGarbledText heuristics, rowLooksLikeDecoyPost post-filter, and fail-closed CommandExecutionError when only decoy rows survive. Includes Adolph decoy fixture and unit tests; messenger bleed behavior from PR #1/#2 unchanged. Co-authored-by: Tom Coustols <tom.coustols@tcdynamics.fr>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Reviewer's GuideThe PR hardens Facebook feed extraction against anti-scrape decoy rows by mirroring expanded readability heuristics in the browser and Node paths, filtering poisoned rows while retaining real posts, and failing with a typed diagnostic when no readable feed content remains; fixture and unit tests cover the new behavior and preserve Messenger bleed handling. Sequence diagram for fail-closed Facebook feed extractionsequenceDiagram
participant CLI
participant Feed as getFacebookFeed
participant Browser as Feed extraction script
participant Filter as Decoy/readability checks
CLI->>Feed: command.func(page, limit)
Feed->>Browser: Extract feed rows
Browser->>Filter: isScrambledOrGarbledText(content)
Filter-->>Browser: Keep readable rows only
Browser-->>Feed: payload.rows
Feed->>Filter: rowLooksLikeDecoyPost(row)
alt readable rows remain
Filter-->>Feed: Return readable rows
Feed-->>CLI: Feed results
else all rows are decoys
Filter-->>Feed: All rows rejected
Feed-->>CLI: CommandExecutionError
end
Flow diagram for Facebook feed decoy filteringflowchart TD
A[Facebook feed DOM] --> B[Browser extraction]
B --> C{Readable post rows?}
C -->|Mixed decoy and real| D[Filter decoy rows]
D --> E[Return readable rows]
C -->|No rows| F[Existing empty-feed handling]
C -->|Decoy rows only| G[Node rowLooksLikeDecoyPost]
G --> H[Throw CommandExecutionError]
H --> I[Retry with diagnostic]
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
lawmight
marked this pull request as ready for review
September 10, 2026 20:41
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 1 issue
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="clis/facebook/feed.js" line_range="89" />
<code_context>
+ return false;
+}
+
+function isScrambledFeedText(text) {
+ const t = cleanFeedText(text);
+ if (!t) return true;
+ if (isDecoyFeedText(t)) return true;
+ if (t.length < 12) return false;
+
+ if (/^[A-Za-z0-9]{10,}:\s/.test(t)) return true;
+
+ const tokens = t.split(/\s+/).filter((word) => word.length > 1);
+ if (tokens.length === 0) return true;
+
+ let scrambled = 0;
+ for (const word of tokens) {
+ const core = word.replace(/[^\w]/g, '');
+ if (!core) continue;
+ if (/\d/.test(core) && /[a-zA-Z]/.test(core) && !/^(19|20)\d{2}$/.test(core)) {
+ if (/[a-zA-Z]\d|\d[a-zA-Z]/.test(core) || /(?=.*[a-z])(?=.*[A-Z])(?=.*\d)/.test(core)) {
+ scrambled += 1;
+ continue;
+ }
+ }
</code_context>
<issue_to_address>
**issue (bug_risk):** A normal two-token post containing one mixed alphanumeric word is always classified as scrambled because one scrambled token reaches the 0.25 ratio threshold; for example, `COVID19 update` has a scrambled ratio of 1/2 and is rejected. This removes legitimate posts containing common product names, model identifiers, or years joined to words.
**Triggers:** When a legitimate post has at least 12 characters, two or more tokens, and one token containing adjacent letters and digits.
**Suggested fix:** Require multiple suspicious mixed tokens or additional soup/entropy evidence before rejecting short content, rather than treating a single mixed alphanumeric token as sufficient.
```suggestion
if (scrambled >= 2 && scrambled / tokens.length >= 0.25) return true;
```
</issue_to_address>Sourcery assessment
Approval pending. 1 finding to address first.
Blocking findings: clis/facebook/feed.js:89
| } | ||
| } | ||
|
|
||
| if (tokens.length >= 2 && scrambled / tokens.length >= 0.25) return true; |
There was a problem hiding this comment.
issue (bug_risk): A normal two-token post containing one mixed alphanumeric word is always classified as scrambled because one scrambled token reaches the 0.25 ratio threshold; for example, COVID19 update has a scrambled ratio of 1/2 and is rejected. This removes legitimate posts containing common product names, model identifiers, or years joined to words.
Triggers: When a legitimate post has at least 12 characters, two or more tokens, and one token containing adjacent letters and digits.
Suggested fix: Require multiple suspicious mixed tokens or additional soup/entropy evidence before rejecting short content, rather than treating a single mixed alphanumeric token as sufficient.
Suggested change
| if (tokens.length >= 2 && scrambled / tokens.length >= 0.25) return true; | |
| if (scrambled >= 2 && scrambled / tokens.length >= 0.25) return true; |
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Live smoke (
opencli facebook feed --limit 5) was exiting 0 with a single anti-scrape decoy row: authorAdolph L'héritageand scrambled/unreadable content. This PR tightens decoy filtering so the extractor either returns ≥1 readable post with a plausible author or fails with a clear diagnostic — never exit 0 with only poisoned rows.Related issue: live smoke 2026-09-11 (tom.coustols, OpenCLI 1.8.8)
Type of Change
Changes
isScrambledOrGarbledText(in-page) +isScrambledFeedText(Node mirror): detect alphanumeric soup, scrambled timestamp prefixes (Mk7sPrtt9B23f81:), low vowel-ratio garbage, and mixed letter-digit tokensisDecoyTextfrom 24→16 chars to catch shorter FB poison tokensrowLooksLikeDecoyPostpost-extraction filter (parallel to existing messenger bleed filter)getFacebookFeedfail-closed: when all rows are decoys →CommandExecutionErrorwith sample author in hint; mixed decoy+real → keep readable rows onlyfeed-adolph-decoy.htmland 5 new unit tests covering Adolph-style decoy rejection, no-rows extraction, row heuristic, fail-closed error, and mixed filteringChecklist
npm test -- clis/facebook/— 107 passed)Messenger bleed regression
PR #1/#2 behavior preserved — all existing messenger bleed tests pass unchanged.
Post-merge verification (KIWI-BACK-TICKET)
After merge + install, run live
opencli facebook feed --limit 5on tom.coustols desk and confirm either readable posts (e.g.LVLUP with Lani) or a typed decoy error — not exit 0 with scrambled junk.Summary by Sourcery
Reject poisoned Facebook feed rows and fail clearly when no readable posts are available.
Bug Fixes:
Enhancements:
Tests:
Summary by cubic
Fixes
opencli facebook feedsilently exiting 0 when Facebook serves only anti-scrape decoy rows (e.g., author "Adolph L'héritage" with scrambled content), so the extractor now returns readable posts or fails with a clear diagnostic. Messenger bleed filtering behavior from earlier PRs is unchanged.Changes
rowLooksLikeDecoyPostpost-extraction filter alongside the existing messenger bleed filter.getFacebookFeednow fail-closes: all-decoy rows throw a typedCommandExecutionErrorwith a sample author; mixed decoy+real rows keep only readable posts.feed-adolph-decoy.htmlfixture and five unit tests covering decoy rejection, no-row extraction, row heuristics, fail-closed errors, and mixed filtering.Written for commit 8b4a87f. Summary will update on new commits.