> ## Documentation Index
> Fetch the complete documentation index at: https://docs.graphext.com/llms.txt
> Use this file to discover all available pages before exploring further.

# fetch_url_content

> Fetch the main text from a web URL, and return its title, author, content, excerpt and domain. 

## Usage

The following example shows how the step can be used in a recipe.

<Accordion title="Examples" icon="code" defaultOpen="true">
  <Tabs>
    <Tab title="Example 1">
      Since the step has no configuration parameter, it's simply

      ```stan theme={null}
      fetch_url_content(ds.article_url) -> (
        ds.article_title,
        ds.article_author,
        ds.article_content,
        ds.article_excerpt,
        ds.article_domain
      )
      ```
    </Tab>

    <Tab title="Signature">
      General syntax for using the step in a recipe. Shows the inputs and outputs the step is expected to receive and will produce respectively. For futher details see sections below.

      ```stan theme={null}
      fetch_url_content(urls: url) -> (
      	title: text,
      	author: category,
      	content: text,
      	excerpt: text,
      	domain: url
      )
      ```
    </Tab>
  </Tabs>
</Accordion>

## Inputs & Outputs

The following are the inputs expected by the step and the outputs it produces. These are generally
columns (`ds.first_name`), datasets (`ds` or `ds[["first_name", "last_name"]]`) or models (referenced
by name e.g. `"churn-clf"`).

<Accordion title="Inputs" icon="right-to-bracket">
  <ParamField path="urls" type="column[url]" required>
    A column of URLs linkling to articles, blog posts or webpages.
  </ParamField>
</Accordion>

<Accordion title="Outputs" icon="right-from-bracket">
  <ParamField path="title" type="column[text]" required>
    A text column containing the extracted article's title.
  </ParamField>

  <ParamField path="author" type="column[category]" required>
    A categorical column containing the extracted article's author.
  </ParamField>

  <ParamField path="content" type="column[text]" required>
    A text column containing the extracted article's main text.
  </ParamField>

  <ParamField path="excerpt" type="column[text]" required>
    A text column containing a summary of the extracted article.
  </ParamField>

  <ParamField path="domain" type="column[url]" required>
    A column containing only the domain of each original URL.
  </ParamField>
</Accordion>

## Configuration

The following parameters can be used to configure the behaviour of the step by including them in
a json object as the last "input" to the step, i.e. `step(..., {"param": "value", ...}) -> (output)`.

<Accordion title="Parameters" defaultOpen="true" icon="sliders">
  This step doesn't expect any configuration.
</Accordion>