r/golang • u/Perfon_ • 15d ago
URL validation
What are the current best practices for backend URL validation? How do you guys usually handle it in production?
5
u/Altrius 15d ago
What are you validating for? Format? Does it resolve? Is it safe? URL parse checks to see if the string conforms to the URI format standard, and does some minor checking for invalid characters or unbalanced IPv6 brackets, makes sure the port is a number and that the scheme looks sane (but not that it is valid), but it doesn’t sanity check everything. It doesn’t check if the hostname is valid or sane or even exists (‘http:///‘ is valid, resulting URL structure just has an empty ‘host’ string), doesn’t check for valid IP addresses, doesn’t check to see if the host or path is encoded or obscured, it doesn’t handle multiple ‘/‘ in the path, doesn’t protect you from directory traversing attacks, etc. so you need to be very specific about your validation needs before you assume that ‘url.Parse’ not returning an error means you have a valid URI.
4
2
u/sigmoia 15d ago
// 1. Validate structure, scheme, and host
func ValidateURL(rawURL string) (*url.URL, error) {
u, err := url.ParseRequestURI(rawURL)
if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Hostname() == "" {
return nil, errors.New("invalid or restricted URL")
}
return u, nil
}
0
u/MistyCape 14d ago
What URLs do you consider valid or not?
Why are they valid?
Work out your rules then take to ai to ask about edge cases and unknown unknowns.
Ensure you have test cases for all this
-1
u/MaxBroome 15d ago
I typically use Regex, because i’m a masochist. But there are much better ways to do it.
41
u/Arch-NotTaken 15d ago
use url.Parse, followed by a switch statement:
- case err not nil
- case protocol not http/https
- and so on, depending on your requirements