TokenList Class Reference

#include <tokenizer.hpp>

Collaboration diagram for TokenList:

Public Member Functions
	TokenList (const Tokenizer &tokenizer, const std::string &text)
size_t	size () const
Token	operator[] (size_t n) const

Private Types
using	TOKEN_REF = typename Tokenizer::TOKEN_REF

Private Attributes
const Tokenizer &	m_tokenizer
const std::string &	m_text
std::vector< TOKEN_REF >	m_tokens

Detailed Description

Definition at line 211 of file tokenizer.hpp.

Member Typedef Documentation

TOKEN_REF

using TokenList::TOKEN_REF = typename Tokenizer::TOKEN_REF

private

Definition at line 213 of file tokenizer.hpp.

Constructor & Destructor Documentation

TokenList()

TokenList::TokenList ( const Tokenizer &  tokenizer, const std::string &  text  )

inline

Definition at line 221 of file tokenizer.hpp.

        : m_tokenizer(tokenizer),
          m_text(text)
    {
        size_t startpos = 0;
        size_t len = m_text.size();
        std::smatch matches;
 
        m_tokens.reserve(len / 5);
 
        while (startpos < len)
        {
            TOKEN_REF tref = m_tokenizer.match(matches, m_text, startpos);
            m_tokens.push_back(tref);
            startpos += tref.len;
        };
    }

Member Function Documentation

operator[]()

Token TokenList::operator[] ( size_t  n ) const

inline

Definition at line 244 of file tokenizer.hpp.

    {
        return Token(m_text, m_tokens[n].pos, m_tokens[n].len, m_tokens[n].type);
    }

size()

size_t TokenList::size ( ) const

inline

Definition at line 239 of file tokenizer.hpp.

    {
        return m_tokens.size();
    }

Referenced by complete_line(), translate(), translate_construct(), translate_data_def(), translate_expression(), translate_instruction(), translate_instruction_param(), translate_list(), translate_macro(), translate_mem_ref(), and translate_record().

Member Data Documentation

m_text

const std::string& TokenList::m_text

private

Definition at line 216 of file tokenizer.hpp.

Referenced by operator[](), and TokenList().

m_tokenizer

const Tokenizer& TokenList::m_tokenizer

private

Definition at line 215 of file tokenizer.hpp.

Referenced by TokenList().

m_tokens

std::vector<TOKEN_REF> TokenList::m_tokens

private

Definition at line 217 of file tokenizer.hpp.

Referenced by operator[](), size(), and TokenList().

---

The documentation for this class was generated from the following file:

• sdk/tools/asmpp/tokenizer.hpp