Semantic localization via the matrix permanent