I think its really about what the developer has built the module to be vs how it can actually be.
I took at look under the hood at the source code and I would suggest the same for more clarity. Also I used the following guide to understand what that code means : https://www.evanmiller.org/nginx-modules-guide.html
The source code for the  ngx_http_map_module is here.
If I look at the static ngx_command_t  ngx_http_map_commands[] = {...} which defines the directives of the module and specifically at the following snippet:
 { ngx_string("map"),
  NGX_HTTP_MAIN_CONF|NGX_CONF_BLOCK|NGX_CONF_TAKE2,
  ngx_http_map_block,
  NGX_HTTP_MAIN_CONF_OFFSET,
  0,
  NULL },
- The first parameter just defines the directive string
 
- The second parameter is interesting. Its the 
type field which is: 
type is a set of flags that indicate where the directive is legal and how many arguments the directive takes.
We see here that the fields defined are NGX_HTTP_MAIN_CONF which makes the directive valid at the http level. But we do not see the corresponding flags NGX_HTTP_SRV_CONF - for server or NGX_HTTP_LOC_CONF for location levels.
So it makes sense that this particular implementation of the ngx_http_map_module only works at the http level.
Now, can it work on the server/location level? Theoretically, it should. But it would need some contribution in the source code. If you think it would make for a useful feature, please do contribute :)