How KerjaCheck works, and how we tested it
KerjaCheck reads a job offer the way a careful friend would: it looks for the specific tricks that Malaysian police and Bank Negara warn about, and it compares the wording with thousands of examples. All of this runs in your browser, so nothing you paste is uploaded.
The pipeline
- Red-flag rules. 15 hand-written rules in English, Bahasa Melayu and Manglish, each based on how job scams and mule recruitment are described in PDRM, NSRC and Bank Negara advisories and Malaysian news. They find the exact phrase that is dangerous, which is what gets highlighted, and explain why.
- Trained classifier. A logistic-regression model on character n-grams (so it copes with spelling like "u", "ur" and "akaun") plus words and word pairs, trained on 960 labelled example messages. It was trained in Python and exported to a 260 KB file that runs in TypeScript; an automated check confirms the two give identical scores.
- Decision. A strong rule match means the message is flagged; the classifier decides whether it looks more like mule recruitment or a job scam. The classifier alone can only raise a caution unless it is very confident, because rules are more reliable on wording it hasn't seen.
- What to do. Every result ends with the next step: Semak Mule (PDRM), the 997 National Scam Response Centre, or SSM e-Search for checking a real company.
How we tested it
A model scored on its own training data proves nothing, so the test messages were written by a separate process that never saw the rules, the model or the training data. Each blind set is balanced between real jobs, job scams and mule offers, and deliberately includes hard cases: real cashier, clerk and freelance jobs that talk about bank accounts and payments, and scams written in formal language with invented company names. Some messages are close paraphrases of offers described in Malaysian news reports. The engine was frozen and committed to Git before each blind run.
Blind set A, engine v1 (first look, before any tuning) (150 messages)
| Method | Scams caught | Real jobs wrongly flagged | Precision | 3-way accuracy |
|---|---|---|---|---|
| Rules only | 58% | 4% | 96.7% | 68.7% |
| Classifier only | 96% | 22% | 89.7% | 87.3% |
| Rules + classifier (shipped) | 97% | 22% | 89.8% | 86.7% |
Blind set A, engine v2 (after error analysis on set A) (150 messages)
| Method | Scams caught | Real jobs wrongly flagged | Precision | 3-way accuracy |
|---|---|---|---|---|
| Rules only | 62% | 0% | 100% | 73.3% |
| Classifier only | 97% | 8% | 96% | 94.7% |
| Rules + classifier (shipped) | 98% | 8% | 96.1% | 94.7% |
Blind set B, engine v2 (final; never used for tuning) (120 messages)
| Method | Scams caught | Real jobs wrongly flagged | Precision | 3-way accuracy |
|---|---|---|---|---|
| Rules only | 51.2% | 17.5% | 85.4% | 61.7% |
| Classifier only | 96.3% | 15% | 92.8% | 87.5% |
| Rules + classifier (shipped) | 96.3% | 22.5% | 89.5% | 87.5% |
Final blind set, shipped engine: where it was right and wrong
| Actual ↓ / KerjaCheck said → | Real job | Job scam | Mule offer |
|---|---|---|---|
| Real job | 31 | 6 | 3 |
| Job scam | 2 | 36 | 2 |
| Mule offer | 1 | 1 | 38 |
On the final set the rules alone miss half the scams, and the classifier alone catches 96% with fewer false alarms than the combination. We still combine them because the rules are what find and explain the exact dangerous phrases; a probability on its own can't tell you why. Reducing the rules' false alarms is the next fix. Each check takes well under a millisecond on a laptop.
Honest limitations
- False alarms on real jobs. On the final blind set, 9 of 40 real jobs got a warning (2 as "be careful", 7 as scam or mule). 7 of the 9 were deliberately hard cases: freelance gigs where the client pays the student a deposit, admin jobs that involve checking customer deposits, and clerks who handle rent or fee collection for a real business. The rules recognise words like "deposit" but not which direction the money flows. Teaching the rules who pays whom is the next fix; we have not tuned on the final set, so this number stays as measured.
- Misses. 3 of 80 scams got through: a fully formal Malay "product evaluator" job using your own funds, a Manglish "collection officer" asking you to withdraw cash received via DuitNow, and an onboarding message asking for IC photos and a selfie.
- All training and test messages are synthetic or paraphrased; we had no access to a dataset of real scam messages received by Malaysian students. Results on real messages will be lower than these numbers.
- The test sets were written by an AI model following public descriptions of scams, not collected from victims.
- Scammers change their wording. Rules need updating, and the classifier should be retrained on messages that users choose to report (with consent) in a future version.
- "No red flags found" never means "safe". KerjaCheck always tells you how to verify.
- Chinese and Tamil scripts are not supported yet; romanised Malay, English and Manglish are.
Open source
The code, the training data generator, both blind test sets and the evaluation scripts are on GitHub so anyone can re-run these numbers.