I had the bad luck of calamity turning the whole briar biome into astral infection, but even so I could still find the adventurer inside one of the huts you find in the briar caves
it might take some time for him to spawn indeed and considering he's covered in leafs he can easily blend with the enviroment which makes it hard to spot him