Choose by scorecard, then confirm with a pilot on your own site
To choose an AI agent for student recruitment, write a weighted scorecard before any demo, score every vendor against it, and run the two best through a short pilot on your real content. The scorecard stops the loudest demo from winning. The pilot shows how each product behaves on your programs, your deadlines and your inquiries.
This guide is for enrollment, admissions and marketing leaders at US colleges and universities who have to defend a vendor decision to a provost, a CIO or a general counsel. It covers eight criteria, a scoring table, red flags, a thirty-day pilot with test questions and the US legal context. If you are earlier in the process, the guide to AI chatbots for student recruitment explains what these tools do. If you are building a formal procurement document, the chatbot RFP checklist for higher education is the companion.
Start with the failure you cannot afford: a wrong answer
An agent should answer only from content you have approved, name the page it used, and say "I don't know" when the content is silent. A prospective student who is told the wrong application deadline, the wrong transfer-credit policy or the wrong cost of attendance has relied on your institution's word.
In the demo, ask the vendor to point the agent at your own site and try:
- a question your pages answer clearly, to see whether the source is shown;
- a question your pages do not answer, to see whether it admits the gap;
- a leading question such as "Will I get a scholarship if my GPA is 3.4?" to see whether it refuses to promise.
Weight this criterion highest. Everything else, from handoff to analytics, assumes the answers are trustworthy.
When the conversation needs a person: qualification and handoff
A good agent asks a few natural questions about program of interest, first-year or transfer status, term and residency, then routes the student to the right counselor with a summary. It should not score students or decide who deserves attention. Ask to see the handoff moment itself: what triggers it, what the counselor receives and what the student is told.
Agree on triggers in advance. Reasonable ones include a request for a human, a complaint, a financial aid question that needs an individual review, a disability or accommodation request and any sign of a welfare concern. Also ask what happens at 11 p.m., when no counselor is online. The honest answer is a queue with a clear promise about when someone will reply.
Integrations: ask the vendor to demonstrate, not describe
Request a live demonstration of how a conversation becomes a record in your CRM and how a campus tour, admitted students day or advising appointment lands on a real calendar. A slide of logos is not evidence. Systems differ, and the work is in the details.
Give each vendor the same short brief: your CRM, your scheduling tool and your consent practices. Ask which fields are written, how duplicates are handled and what the team sees if a write fails. Remember that most first-year applicants file through the Common App or an institutional application, so your agent supports the steps before and after that submission rather than replacing it.
Data, language, review, channels and exit
These five criteria are quiet in a demo and loud in a security review.
Privacy and hosting. Ask where data is stored, who the sub-processors are, whether your content and conversations train any model and what the retention settings are. Ask for the security documentation your IT office usually requests. If the agent will see student education records, involve your FERPA officer early.
Transparency. The agent should say it is an automated assistant, at the start.
Multilingual and after-hours. Test the languages your applicants actually use. A Spanish-language question about financial aid is a better test than a long list of languages.
Conversation review. Counselors should be able to read transcripts, see unanswered questions and spot gaps in your web content.
Channels. Start with the website. Treat text messaging or WhatsApp as a separate decision with its own consent review.
Pilot, cost and exit. Ask for a time-boxed pilot, an itemized explanation of what the total cost covers and a written answer on exporting your conversations and configuration if you leave.
The scorecard
Score each criterion from 0 to 5, multiply by the weight and compare totals. The weights are a suggestion. Adjust them for your institution, then lock them before demos begin.
| Criterion | Suggested weight | Evidence to ask for |
|---|---|---|
| Answer grounding, sources, refusal | 20% | Live test on your site, visible sources |
| Handoff to counselors | 15% | Trigger list, summary shown, after-hours rule |
| CRM and calendar integration | 15% | Demonstration on your systems |
| Privacy, hosting, no training on your data | 15% | Security documents, sub-processor list, DPA |
| Multilingual and after-hours | 10% | Tests in your applicant languages |
| Transcript review and analytics | 10% | Sample dashboard with real transcripts |
| Pilot terms and cost transparency | 10% | Written pilot plan, itemized cost |
| Portability and exit | 5% | Export format and exit clause |
Have a counselor, an IT representative and someone from marketing score separately, then compare. Large gaps between scorers point to the questions worth asking again.
Red flags
Walk away, or at least pause, when you see any of these:
- the vendor will not run the agent on your content before you sign;
- answers never show a source and never say "I don't know";
- the agent predicts admission or aid outcomes;
- integration claims stay general when you ask for a demonstration;
- the vendor cannot state in writing where data is hosted;
- your content or conversations may be used to train models;
- there is no pilot, no export right and no exit clause;
- pricing relies on a metric nobody can explain.
US legal and accreditation context
There is no single national privacy law for this use, so check several layers. FERPA governs education records at institutions that receive federal funds, and an agent that touches those records needs to fit your FERPA practices. State privacy and breach-notification laws vary, so your counsel should review the vendor's terms against the states where your applicants live. The FTC enforces against deceptive claims, which is a reason to keep what the agent says about costs, outcomes and aid within what you can substantiate.
Outreach by text message or WhatsApp deserves a separate review of consent rules with counsel, including the federal rules on automated messages. Do not assume that a website chat opt-in covers another channel.
Regional and programmatic accreditors also expect accurate public information about programs. If your agent describes a program or its outcomes, the statement should match your catalog and your disclosures.
A thirty-day pilot with test questions
Pick one program area and two actions, such as answering questions and booking campus tours. Define success first, for example fewer inquiries left unanswered overnight and cleaner CRM records.
- Week one. Load approved content, set handoff triggers, connect CRM and calendar.
- Week two. Counselors try to break the agent with the questions below. Fix content gaps.
- Weeks three and four. Launch on selected pages, review transcripts weekly and log handoffs, bookings and unanswered questions.
Questions to include:
- "Do you accept AP credit toward this major?"
- "What is the priority deadline for fall, and for transfer students?"
- "Can my parents join the campus tour?"
- "I'm an international student. Can I work on campus?" (point to published guidance, hand off)
- "Will I get a full scholarship?" (must refuse to promise)
- "I'm struggling and don't know if college is for me." (human handoff)
- A Spanish-language question sent late at night.
Then score the pilot with the same scorecard. To build the shortlist, see the comparison of AI chatbots for higher education and the comparison of AI agents for student recruitment.
Skolbot Chat is a website agent that answers from your institution's own content, qualifies prospective students, books tours and appointments and pushes the record to your CRM, with WhatsApp as an optional module. Put it through the same scorecard as everyone else.


