Technically it can be both.
Chalk (uncountable)- The material or substance chalk.
Chalk (countable)- A piece or stick of chalk.
If you said "chalk" or "a chalk" I think a native speaker would easily assume either of those definitions. Calling it a stick or a piece instead would be preferred, but I have heard educated native speakers saying just "chalk" like that. Maybe it's an informal shorthand.
But chalk is probably a special case, because maybe one of the most frequently seen uses of chalk is in the form of chalk sticks, like in school). If you tried to apply this rule to any substance, without a proper quantifier (drop, piece, etc.), there is a high chance it'll sound super weird and just wrong. Like gasoline. It's uncountable. You can't say "a gasoline". But "a chalk"? Semantically, no. But technically, yes.