Y7 CLIP Token Counter

Y7Nodes_CLIP_TokenCounter

CLIP Token Counter<div style="font-size: 0.8em"><div id="Y7_shortdesc" style="margin-bottom: 15px;">Counts tokens in a text using the CLIP tokenizer.</div></div><div style="font-size: 0.8em"><div style="margin-bottom: 8px; margin-left: 0px; font-size: 12px;">Up to the first 77 tokens will be displayed, with any overflow tokens shown below.</div></div><div style="font-size: 0.8em"><div style="margin-bottom: 8px; margin-left: 0px; font-size: 12px;">The actual limit is 77 tokens, including two special tokens: <code style="border: 1px solid #666; border-radius: 3px; padding: 0px 1px; font-family: monospace; display: inline-block;">&lt;|startoftext|&gt;</code> (Beginning-of-Sequence) and <code style="border: 1px solid #666; border-radius: 3px; padding: 0px 1px; font-family: monospace; display: inline-block;">&lt;|endoftext|&gt;</code> (End-of-Sequence).</div></div><div style="font-size: 0.8em"><div style="margin-bottom: 8px; margin-left: 0px; font-size: 12px;">The final token in the sequence will be shown, along with a brief context of words leading up to it.</div></div><div style="font-size: 0.8em"><div style="margin-bottom: 8px; margin-left: 0px; font-size: 12px;">The <code style="border: 1px solid #666; border-radius: 3px; padding: 0px 1px; font-family: monospace; display: inline-block;">&lt;/w&gt;</code> marker indicates a word boundary, typically where a space followed the word in the original text.</div></div><div style="font-size: 0.8em"><div style="margin-bottom: 8px; margin-left: 0px; font-size: 12px;">Inputs:</div></div><div style="font-size: 0.8em"><div style="margin-bottom: 8px; margin-left: 20px; font-size: 12px;">- text_in: Any text (string) input.</div></div><div style="font-size: 0.8em"><div style="margin-bottom: 8px; margin-left: 20px; font-size: 12px;">- show_tokens: Displays tokenized version of text (requires re-run).</div></div><div style="font-size: 0.8em"><div style="margin-bottom: 8px; margin-left: 20px; font-size: 12px;">- tokens_per_line: Number of token words per line (requires re-run).</div></div><div style="font-size: 0.8em"><div style="margin-bottom: 8px; margin-left: 0px; font-size: 12px;">Output:</div></div><div style="font-size: 0.8em"><div style="margin-bottom: 8px; margin-left: 20px; font-size: 12px;">- text_out: A pass-through output for the input string</div></div><div style="font-size: 0.8em"><div style="margin-bottom: 8px; margin-left: 0px; font-size: 12px;">Other Widgets:</div></div><div style="font-size: 0.8em"><div style="margin-bottom: 8px; margin-left: 20px; font-size: 12px;">- font_size: Change font size used in the text widget. </div></div><div style="font-size: 0.8em"><div style="margin-bottom: 8px; margin-left: 20px; font-size: 12px;">- Copy Text: Copy contents of the text widget.</div></div><div style="font-size: 0.8em"><div style="margin-bottom: 8px; margin-left: 0px; font-size: 12px;"></div></div><div style="font-size: 0.8em"><div style="margin-bottom: 8px; margin-left: 0px; font-size: 12px;">Note: Longer prompts are supported, but how they are handled depends entirely on the specific implementation of the model and tokenizer. Some implementations may truncate, segment, or otherwise process longer inputs differently.</div></div>

Pack: Y7Nodes for ComfyUI

custom_nodes.ComfyUI-Y7Nodes