The evaluation checklist
Ask all of us the same questions.
We do not build against a competitor’s feature list. We build against the problems an MSP actually has, so instead of a page telling you we win, here is the sheet we would want if we were buying. Take it to every vendor you are talking to, including us.
Our column is filled in, and it includes the rows where the honest answer is not yet. There are eleven of those across 73 questions, and the last group is nothing but hard ones about us. A checklist where every answer favors whoever wrote it is not worth printing.
Section one
The business questions.
For the owner. These decide whether the thing makes you money, and most of them have nothing to do with features.
Adoption
Does it work inside the tools your engineers already have open?
Every tool an MSP abandons was abandoned because it was one more place to go.
How long from signature to a team using it daily?
A twelve-week implementation is a twelve-week delay on the return.
How many hours of your own team's time does connecting it take?
The honest number, not the vendor's. This is the line item that stalls a rollout.
What retraining does it require?
Anything that needs a training program will be used by the two people who attended.
- Yes: Proof of concept in a week, full onboarding inside a couple of months
- Yes: Four to six hours total: API members and permissions, documentation access, picking the pilot group
The business view
Can you see what each client actually consumes to serve?
Under a fixed fee, effort per account is the thing that decides whether it makes money, and it is invisible until a renewal.
Does it show which accounts consume more than they are entitled to?
Over-consumption against the agreement is the fixed-fee form of revenue leakage.
Does it identify work performed outside the agreement?
Out-of-scope work absorbed silently is margin leaving without anyone seeing it go.
Can you prove the value to a client without building the report yourself?
If producing the evidence is a project, it will not happen every quarter.
- Yes: Volume, effort and utilization per client. We deliberately hold no rates and display no financial data, so you apply your own numbers
- Partial: Client Health arrives built and deflection is countable in your PSA. A client-facing render of the value report is not built yet
Ecosystem and independence
Does the vendor own any of the systems it has to read?
The single most useful question on this sheet, and the one a PSA vendor cannot answer the way you want.
Does it read your documentation platform even if a competitor owns it?
ConnectWise will not build for IT Glue, because IT Glue is Kaseya's. Your stack does not respect those lines and neither should your AI.
Does it support more than one PSA?
A tool welded to one PSA is a bet on never changing your mind.
If you change PSA, does it come with you?
The average MSP changes PSA at least once. A PSA-native tool is something you leave behind, along with what it learned.
If your PSA vendor bundles this free, what happens to you?
Ask every vendor, and ask it about themselves. The answer tells you whether you are buying a product or a feature waiting to be absorbed.
Is the AI roadmap independent of selling you the vendor's other products?
A platform vendor's AI exists to make their platform stickier. That is a reasonable thing for them to want and a bad thing for you to depend on.
- Yes: We own none of them, so we have no reason to prefer one
- Yes: ConnectWise tickets, IT Glue pages and SharePoint searched together
- Yes: ConnectWise Manage, Autotask, HaloPSA, ServiceNow
- Yes: Cross-platform by design, not by roadmap
Your brand
Do your clients see your name on it, or the vendor's?
The application their staff open every day is either your product or somebody else's.
Can the tone and behavior be set per client?
A law firm and a construction company do not want the same voice answering their people.
Will the vendor market to your clients' staff?
Every MSP has been disintermediated by a supplier that started out helpful.
- Yes: Agent-X can be branded to your MSP. The signed binary is still ours
- Yes: We do not, and there is no end-user marketing anywhere on our site
Control
Can you decide what the AI may do unattended, per client?
You are taking a risk on your clients' behalf, in their tenancy, under your name.
Can approvals route to someone other than your engineers?
Your engineers were never the right approver for who may join a shared mailbox.
Can a capability be switched off entirely, not just gated?
"We do not want this" should never have to masquerade as an approval rule.
Is every action recorded, with who asked and who approved?
The first thing anyone asks after an agent does something surprising.
- Yes: Twelve decisions, six positions, set once and overridden per client
- Yes: To the resource owner, in the app
Security and data
Will they tell you which models they run on?
Most vendors will not. You are going to be asked this by your own clients, so you need an answer.
Is your data contractually excluded from training their models?
Not a promise in a sales call. A term you can point at.
Is there an independent audit, and of what type?
A Type 2 tests whether controls actually operated over time. A Type 1 tests whether they existed on the day somebody looked.
Do you get written confirmation your data was destroyed when you leave?
Your own clients' auditors will ask you this about your vendors.
Is there a security page you can forward to your own clients?
Their questions arrive at your desk. Writing the answers yourself is a tax.
- Yes: Anthropic, OpenAI and Groq, named on our security page
- Yes: Zero-retention endpoints, and the MSA binds our subprocessors to the same
- Yes: SOC 1 Type 2 and SOC 2 Type 2, both complete, reports under NDA
- Yes: Deletion inside 60 days and a data destruction letter, in the MSA
Commercials
Is the price published?
A vendor who hides pricing is a vendor still deciding what to charge you.
What is the pricing unit, and what is not included?
Per engineer, per seat and per endpoint are three completely different bills.
Can you start on a subset of your desk rather than all of it?
A platform-wide commitment before anyone has used it is how shelfware gets bought.
- Yes: Assist-X $55 per engineer per month, XOP Command $500 per month, minimums published as counts, Agent-X not yet priced
- Yes: A paid pilot on a pilot group, typically twenty engineers
Section two
The functional questions.
For whoever runs the service desk. These decide whether it survives contact with a Tuesday.
Knowledge and search
Does it search every ticket your team has already resolved?
The answer to most tickets exists. Finding it cost more than guessing.
Does it search your documentation platform, not just tickets?
Half your answers are in IT Glue, Hudu or SharePoint, not the PSA.
Does it work if your documentation is incomplete?
It is incomplete. Every MSP's is. A tool that needs it fixed first never starts.
Does it write the missing documentation when it finds a gap?
The only moment anyone can see a specific hole is when an engineer needs the page.
Does the draft land back in your documentation system for review?
Documentation that lives in the AI's memory is not documentation.
The ticket itself
Does it transcribe calls hands-free into the ticket?
The clerical half of a service call, and the half that gets skipped when the queue is long.
Does it capture time entries as the work happens?
Time reconstructed at five o'clock is always lower than the work actually done.
Does it set type, subtype and priority correctly?
Classification drives your reporting. Wrong classification is worse than none.
Does anything write to your PSA without a human approving it?
The first question every service manager asks, and the answer decides whether engineers trust it.
Does it help write the response to the client?
Tone matters differently for a managing partner and a graduate.
- Yes: Nothing posts until an engineer edits and approves the draft
What it notices
Is every ticket analyzed, or a sample?
Claim the denominator. A manager spot-checks a handful a month.
Does it flag chronic issues across time?
The fourth time is a project conversation, not another repair.
Does it recognize a multi-user event?
Three tickets that look unrelated are one outage.
Does it flag VIP contacts before the ticket goes wrong?
The managing partner, not a graduate. Handled accordingly.
Does it recommend escalation, with the reasons in plain language?
A risk score nobody can interrogate gets ignored within a month.
Can you change what counts as urgent, and re-score against it?
A business living on SLA attainment and one living on executive relationships do not want the same tickets raised.
Does it measure sentiment on every interaction?
Surveys catch the furious and the delighted, and arrive late.
- Yes: Eight weights you set, and the queue re-scores
Revenue
Does it check requests against the client's agreement in real time?
Before an engineer spends the afternoon on work nobody is paying for.
Can an engineer see what that client actually bought?
An engineer who cannot see the agreement guesses at it.
Does it surface project and upgrade opportunities with evidence?
An opportunity without the reasoning attached does not survive a sales conversation.
Do those opportunities land in the PSA as work assigned to someone?
Spotting it and acting on it should be the same event.
Is there a dedicated revenue operations view?
Leakage and opportunity as numbers you manage, rather than events you happen to catch.
- Partial: Both halves run inside Assist-X today. The aggregate view is not built
Your team
Does it measure engineer quality, not just throughput?
Throughput tells you who is busy. It does not tell you who is good.
Are skills derived from work actually done?
A skills matrix somebody maintained once is fiction within a quarter.
Does it help schedule work to the right engineer?
Skills, availability, client familiarity and workload, not just who is free.
Can it fill an engineer's day, not just assign a ticket?
Filling a day is the utilization number, and utilization is margin.
Your clients
Can end users resolve common requests themselves?
Password resets and MFA lockouts are the highest-volume, lowest-skill work you carry.
Can a locked-out user with a dead authenticator get back in unaided?
The specific case that otherwise costs an identity check over the phone and an engineer's time.
Does that go past break-fix into everyday productivity work?
Access to a channel, a list or a shared mailbox is most of what actually arrives.
Can end users see their own ticket history and talk to a person?
AI first, never AI only. The way out has to be one tap.
When it cannot help, does the engineer inherit what was already tried?
Otherwise the escalation restarts the diagnosis and the client repeats themselves.
Are inbound calls answered and turned into classified tickets?
A voicemail is a task for somebody else.
Does it run on macOS at parity with Windows?
One client with eight Macs is enough to make this a problem you have to explain.
- Yes: A Temporary Access Pass signs them in without password or existing MFA
- Partial: 54 end-user tools on Windows, 46 on macOS. Identity, access and tickets identical; some local remediation is Windows only
The rest of your stack
Can it act on a machine other than the one asking?
Half of what a service desk does is to somebody else's laptop.
Does it read RMM data into the ticket for context?
Answering with what the machine is doing beats asking the user to describe it.
Does it know hardware age and lifecycle?
Lifecycle risk becoming a quotable project, from asset data rather than inference.
Can it reach the applications your clients actually run?
Microsoft 365 is where most of the work is. Almost none of your clients stop there.
- Yes: Six RMM tools, behind engineer approval, never self-service
- Not yet: On the roadmap, not shipped
- Yes: Through ScalePad Lifecycle Manager, where you have it
- Partial: Microsoft 365 and the desktop in the October release. Other applications are the second release
Maturity, and what is not built
Is every product on the sheet generally available?
Ask it plainly. A roadmap demonstrated as a product is the oldest trick in this industry.
Are the deflection numbers measured, or projected?
Both are useful. Being told which is which is the part that matters.
Is there a named reference customer you can call?
Not a logo on a slide. A person who will take the call and say what went wrong too.
Can you build workflows that have nothing to do with IT?
Purchase orders, time off, onboarding. The same approval routing, pointed somewhere else.
Does it triage suspected phishing reported by end users?
A habit you want from your clients' staff, that currently costs your desk an hour a day.
Can you author unbounded automations of your own?
Powerful, and the thing most likely to do something nobody intended.
How big is the company, and does that matter to you?
Small means the person answering your technical question decided the answer. It also means fewer people.
- Partial: Assist-X, Voice-X and XOP Command are. Agent-X enters beta in October 2026
- Partial: None are measured yet. 20 to 40% of level one is our estimate, the higher figures are projections, and the beta cohort is what turns either into data
- Not yet: Case studies are in progress and not yet published
- Not yet: The direction, not the October release
- Not yet: Coming
- Partial: Ships off by design. Self-service authoring is a later release
- Yes: Small, and we say so on our about page rather than implying otherwise
The question worth asking every vendor
“What happens when ConnectWise ships this for free?”
It is the right question and you should put it to all of us. Our answer has three parts, and none of them is that it will not happen. The PSA vendors will absolutely add AI to their own products. That is a given.
They will move slower
A platform vendor with tens of thousands of MSPs on it does not ship something half-finished into their own ticketing system. They are right to be careful, and careful is slow.
They will build for their own ecosystem
ConnectWise will not build for IT Glue, because IT Glue belongs to Kaseya. Halo will build for Halo. Why would any of them extend their roadmap to a competitor's documentation platform? We will read ConnectWise tickets, IT Glue pages and SharePoint together, because we do not own any of them.
Their version cannot follow you
The day you move from ConnectWise to Halo, a PSA-native tool is something you leave behind along with the historical data it learned from. Ours moves with you. That is not a feature we shipped, it is the consequence of not being owned by your PSA.
Being independent of your PSA is the whole position. Nobody who owns your ticketing system is going to build across it.
Put the same question to a vendor built on a single PSA and listen for whether they have an answer at all. It is also worth asking us what we would do, and we will tell you.
Score us on it, in front of us.
The fastest way to use this sheet is to make a vendor walk it with you, line by line, and watch which rows they would rather move past.