On the other hand, if you made something you could give to anyone who applied and was little to no work for you how many good candidates would you pick up that you wouldn't have given a phone screen to?
I think that varies quite a lot based on company profile and role-type.
We phone screen anyone who has a resume that's vaguely relevant. If you're applying for a software engineering role and have never done any development, then we'll reject straight away, but that almost never happens.
We simply don't have a high enough profile to receive a flood of applications that we need to filter before we phone screen.
And for a software engineering role, evaluating the work test takes as much as the phone screen (sometimes more). We're not simply checking whether you can produce the right output for FizzBuzz (checking that could be automated), we're looking at your choice of algorithm, design trade-offs, unit testing approach, etc - things that require a human to be involved.
Different companies and different roles will have different time investments.
A CTF style task can be evaluated more simply (at least on first pass) - did you capture the flag.
And if you are looking for a few amazing candidates form amongst a mass of poor ones, then you can afford to optimise your process so that you test every candidate at as low a cost as possible, even if that causes some candidates to self-select out.
