Skip to content

Browse a host's folder tree, list every host's root, or search folders and pages across the census -- each returned folder carrying a preview of its direct URLs

Request

GET /v3/site-censuses/{censusId}/url-folders

In both modes every counted folder row nests a preview of its own DIRECT child URLs in urls (capped at urlsPerFolder, with the uncapped total in matchedUrlCount), so one request returns a fully renderable tree level -- folders and the pages inside them -- without a follow-up call to folders/urls. That endpoint remains the way to page through a folder whose preview was truncated; it lists in the same default (URL, id) order, so the preview's URLs reappear there in the same relative order. It lists every direct URL of the folder without this endpoint's view filtering, so its rows and totals can exceed the preview's matchedUrlCount (e.g. on censuses with IGNORE patterns or in excluded views).

Browse mode (no searchPattern): With host, one level of that host's folder tree; without host, the root folder of every host on the census. Every row is a counted row, so every row carries the preview: all of its in-view direct URLs, up to the cap. Counts always cover each folder's entire subtree, whether they are read from the persisted folder rollup or computed live, so a folder's totalUrls does not change when the census finalizes.

Search mode (searchPattern supplied): A folder is a hit when its OWN name (folderName; a host's root folder is named "/" and is instead findable by its hostname) contains the pattern, or when it DIRECTLY contains a URL whose own page name -- the last segment of its path, not the folders above it -- contains the pattern. Hits come back in folders with matched=true, each nesting its matching direct URLs in urls. Neither kind of match inherits: searching "shoes" does not return every page under a /shoes/ folder (the folder is a name hit, but its urls preview lists only direct children whose own name matches), and a page named shoes.html puts its containing folder on the page without making that folder's name a match.

folders additionally carries the unmatched ancestor folders needed to render a connected tree path, with matched=false and no urls/matchedUrlCount. Every hit is connected back to the root (or to parentFolderPath when supplied). parentFolderPath still requires host, but host on its own is a whole-host search.

Pagination counts matched folders only: totalCount is the number of matching folders, and one page is pageSize matched rows. Nested urls entries and ancestor context rows are additional and never counted. A matching URL appears exactly once across the whole pagination -- nested under its containing folder, on the page where that folder sorts.

Where the counts come from: the excluded view (returnExcludedOnly=true) is always computed live from URLs, and so is the included view whenever the census is running, has IGNORE patterns configured, or has not finished folder aggregation; otherwise the included view is served from the persisted folder rollup. A search result's folder counts always describe the folder's whole subtree on both paths -- they are never restricted to matching URLs -- so the same folder reports the same totalUrls whether it was reached by browsing or by searching. Persisted counts still bucket status codes slightly differently from live ones: live treats any status other than 0 as visited, while the persisted unknown-status bucket additionally absorbs missing, 1xx and 6xx+ codes and reports them under urlsToVisit. visitedUrls + urlsToVisit always equals totalUrls on both paths.

Security
API_Key
Path
censusIdinteger, (int64)required

Unique identification number for a site census

Query
hoststring

The URL hostname to scope folder results. Omit (or send blank) to list the root folder of every host on the census (initial browse view; parentFolderPath must then also be omitted, and each root row carries host-wide counts).

parentFolderPathstring

The normalized parent folder path key; returns immediate child folders under this parent. Coupled to host: requires host (400 if supplied without it), and defaults to the root path "/" when host is supplied without it. Blank counts as omitted. Caller variations are normalized the same way ingestion normalizes stored keys (lowercased, repeated slashes collapsed, trailing slash dropped); a path deeper than 50 folders, with any single folder name longer than 512 characters, or longer than 2048 characters overall cannot name a real folder and is a 400 rather than being answered with a shallower folder's children.

searchPatternstring

Case-insensitive substring matched against a folder's own name (or, for a host's root folder, its hostname) and against a page's own name -- the last path segment. For page names, glob * is treated as the dynamic-segment symbol, so * finds dynamically-segmented pages; folder names are raw path text, so a * in the pattern only matches a literal * in the name. Blank counts as omitted (browse mode). May be combined with or without host.

returnExcludedOnlyboolean

Indicates that the results will only contain excluded folder results: external folders (a non-included baseDomain) reachable via a discovered link, or internal folders matching an IGNORE filter pattern.

pageinteger, >= 0

Number of results page of an endpoint with paginated results

Default:0
pageSizeinteger, [ 1 .. 10000 ]

Page size, counted in folder rows (matched rows in search mode). Up to 10000 when searchPattern is supplied; requests without a searchPattern are limited to 1000 (400 above that). Nested urls and ancestor context rows are additional, so worst-case response size scales with pageSize * urlsPerFolder.

Default:100
urlsPerFolderinteger, [ 0 .. 100 ]

The maximum number of direct-child URLs nested in each counted folder row's urls array. The uncapped in-view total is always reported in matchedUrlCount, so a truncated folder can be paged onward through the folders/urls endpoint, which lists in the same default (URL, id) order (unfiltered by this endpoint's view, so it may interleave additional URLs). 0 disables previews entirely: urls and matchedUrlCount are then absent from every row, matching the pre-preview response shape.

Default:50
sortBystring

Results are ordered by this column and then by (host, folderPath), so a page is stable even when the sort column ties. On the hostless root listing FOLDER_NAME orders by host: every row there is some host's "/" folder, so folderName is the same literal on every row and carries no ordering. Nested urls previews are not governed by this sort; they always list in (URL, id) order, matching the folders/urls endpoint's default.

Default:"FOLDER_NAME"
Enum:"FOLDER_NAME""TOTAL_URLS""URLS_TO_VISIT""VISITED_URLS""BROKEN_URLS""LAST_FOUND"
sortDescboolean

Controls sorting order

Default:false
GET
/v3/site-censuses/{censusId}/url-folders
curl -i -X GET \
  'https://api.observepoint.com/v3/site-censuses/{censusId}/url-folders?host=string&parentFolderPath=string&searchPattern=string&returnExcludedOnly=true&page=0&pageSize=100&urlsPerFolder=50&sortBy=FOLDER_NAME&sortDesc=false' \
  -H 'Authorization: YOUR_API_KEY_HERE'

Responses

Successful operation

Bodyapplication/json
metadataobjectrequired

Common metadata for iterable

foldersArray of objects, >= 0 itemsrequired
Response
{ "metadata": { "pagination": {} }, "folders": [ {} ] }