Getting started with Haskell Language Overloaded Literals Foldable Traversable Lens QuickCheck Common GHC Language Extensions Free Monads Type Classes IO Record Syntax Partial Application Monoid Category Theory Lists Sorting Algorithms Type Families Monads Stack Generalized Algebraic Data Types Recursion Schemes Data.Text Using GHCi Strictness Syntax in Functions Functor Testing with Tasty Creating Custom Data Types Reactive-banana Optimization Concurrency Function composition Databases Data.Aeson - JSON in Haskell Higher-order functions Containers - Data.Map Fixity declarations Web Development Vectors Cabal Type algebra Arrows Typed holes Rewrite rules (GHC)Date and Time List Comprehensions Streaming IO Google Protocol Buffers Template Haskell & QuasiQuotes Phantom types Modules Tuples (Pairs, Triples, ...)Graphics with Gloss State Monad Pipes Infix operators Parallelism Parsing HTML with taggy-lens and lens Foreign Function Interface Gtk3 Monad Transformers Bifunctor Proxies Applicative Functor Common monads as free monads Common functors as the base of cofree comonads Arithmetic Role Arbitrary-rank polymorphism with RankNTypes GHCJS XML Reader / ReaderT Function call syntax Logging Attoparsec zipWithM Profunctor Type Application

Parsing HTML with taggy-lens and lens

Extract the text contents from a div with a particular id

Taggy-lens allows us to use lenses to parse and inspect HTML documents.

#!/usr/bin/env stack
-- stack --resolver lts-7.0 --install-ghc runghc --package text --package lens --package taggy-lens

{-# LANGUAGE OverloadedStrings #-}

import qualified Data.Text.Lazy as TL
import qualified Data.Text.IO as T
import Text.Taggy.Lens
import Control.Lens

someHtml :: TL.Text
someHtml =
    "\
    \<!doctype html><html><body>\
    \<div>first div</div>\
    \<div id=\"thediv\">second div</div>\
    \<div id=\"not-thediv\">third div</div>"

main :: IO ()
main = do
    T.putStrLn
        (someHtml ^. html . allAttributed (ix "id" . only "thediv") . contents)

Filtering elements from the tree

Find div with id="article" and strip out all the inner script tags.

#!/usr/bin/env stack
-- stack --resolver lts-7.1 --install-ghc runghc --package text --package lens --package taggy-lens --package string-class --package classy-prelude
{-# LANGUAGE NoImplicitPrelude #-}
{-# LANGUAGE OverloadedStrings #-}

import ClassyPrelude
import Control.Lens hiding (children, element)
import Data.String.Class (toText, fromText, toString)
import Data.Text (Text)
import Text.Taggy.Lens
import qualified Text.Taggy.Lens as Taggy
import qualified Text.Taggy.Renderer as Renderer

somehtmlSmall :: Text
somehtmlSmall =
    "<!doctype html><html><body>\
    \<div id=\"article\"><div>first</div><div>second</div><script>this should be removed</script><div>third</div></div>\
    \</body></html>"

renderWithoutScriptTag :: Text
renderWithoutScriptTag =
    let mArticle :: Maybe Taggy.Element
        mArticle =
            (fromText somehtmlSmall) ^? html .
            allAttributed (ix "id" . only "article")
        mArticleFiltered =
            fmap
                (transform
                     (children %~
                      filter (\n -> n ^? element . name /= Just "script")))
                mArticle
    in maybe "" (toText . Renderer.render) mArticleFiltered

main :: IO ()
main = print renderWithoutScriptTag
-- outputs:
-- "<div id=\"article\"><div>first</div><div>second</div><div>third</div></div>"

Contribution based upon @duplode's SO answer

Contributors

Topic Id: 6962

Example Ids: 23505,24190

This site is not affiliated with any of the contributors.