As other have explained you cannot block URLs on packet level where router L7 filters work.
It is possible if run a http(s) proxy and force users to use by blocking outgoing ports 80/443 if internal source IP is not from proxy.
The proxy works on http(s) instead of packet level and hence has access to destination URLs.
If you want to filter URLs in all cases, including TLS1.3 and plugging DoH and https based VPNs, you need a proxy terminating ssl/https and re-encrypting towards clients. With all clients having a certificate installed so they accept the connections although re-encrypted by the proxy. This is often called “Corporate MITM” and is what many Enterprise and Education networks do. And the only way to really lock things down.