Skip to content

GitLab

Commit c60deaea authored by Daniel Brötzmann's avatar Daniel Brötzmann
Browse files
Options

[syntax_highlight] Fix deprecation warnings, fix pylint errors 

This fixes some deprecation warnings related to Gtk.Color and restructures
code and formatting according to pylint suggestions
parent 845faff2
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 491 additions and 489 deletions
syntax_highlight/README.md
View file @ c60deaea
# Syntax Highlighting Plugin for Gajim
[Gajim](https://gajim.org) Plugin that highlights source code blocks in chatbox.
[Gajim](https://gajim.org) Plugin that highlights source code blocks in the chat window.
## Installation
The recommended way of installing this plugin is to use
Gajim's Plugin Installer.
The recommended way of installing this plugin is to use Gajim's Plugin Installer.
For more information and instruction on how to install plugins manually, please
refer to the [Gajim Plugin Wiki seite](https://dev.gajim.org/gajim/gajim-plugins/wikis/home#how-to-install-plugins).
refer to the [Gajim Plugin Wiki site](https://dev.gajim.org/gajim/gajim-plugins/wikis/home#how-to-install-plugins).
## Usage
This plugin uses markdown-style syntax to identify which parts of a message
should be formatted as code in the chatbox.
should be formatted as code in the chat window.
```
Inline source code will be highlighted when placed in between `two single
Inline source code will be highlighted when placed in between `two single
back-ticks`.
```
......@@ -35,8 +34,8 @@ i.e. there must be a newline here.
```
````
In case no languge is specified with the opening tag or the specified language
could not be identified, the default languge configured in the settings is
In case no language is specified with the opening tag or the specified language
could not be identified, the default language configured in the settings is
used.
You can test it by copying and sending the following text to one of your
......@@ -53,20 +52,19 @@ using the plugin.)
## Relation to XEP-0393 - 'Message Styling'
https://xmpp.org/extensions/xep-0393.html#pre-block
In [XEP-0393](https://xmpp.org/extensions/xep-0393.html),
the back-tick based syntax is defined as markup for preformatted
text blocks, respectively inline performatted text.
Formatting of such text blocks with monospaced fonts is recommended by the XEP.
text blocks, respectively inline preformatted text.
Formatting of such text blocks with mono-spaced fonts is recommended by the XEP.
By using the same syntax as defined in XEP-0393 XMPP clients with only XEP-0393
support but without syntax highlighting can at least present their users blocks
of pre-formatted text.
of preformatted text.
Since text in between the back-tick markers is not further formatted by this
plugin, it can be considered "pre-formatted".
plugin, it can be considered "preformatted".
Hence, this plugin is compatible to the formatting options defined by XEP-0393,
[section 5.1.2, "Preformatted Text"](https://xmpp.org/extensions/xep-0393.html#pre-block)
and [section 5.2.5, "Preformatted Span"](https://xmpp.org/extensions/xep-0393.html#mono).
......@@ -84,7 +82,7 @@ including default language, style, font settings, background color and formattin
of the code markers.
In the configuration window, the current settings are displayed in an
interactive preview pannel. This allows you to directly check how code would
interactive preview panel. This allows you to directly check how code would
look like in the message
window.
......@@ -117,7 +115,7 @@ in a terminal to display the debug messages.
## Credits
Since I had no experience in writing Plugins for Gajim, I used the
[Latex Plugin](https://trac-plugins.gajim.org/wiki/LatexPlugin)
[Latex Plugin](https://dev.gajim.org/gajim/gajim-plugins/wikis/LatexPlugin)
written by Yves Fischer and Yann Leboulanger as an example and copied a big
portion of initial code. Therefore, credits go to the authors of the Latex
Plugin for providing an example.
......
syntax_highlight/chat_syntax_highlighter.py
View file @ c60deaea
......@@ -4,15 +4,14 @@ import pygments
from gi.repository import Gtk
from gajim.plugins.helpers import log
from .gtkformatter import GTKFormatter
from .types import MatchType, LineBreakOptions, CodeMarkerOptions
from syntax_highlight.gtkformatter import GTKFormatter
from syntax_highlight.types import MatchType
from syntax_highlight.types import LineBreakOptions
from syntax_highlight.types import CodeMarkerOptions
log = logging.getLogger('gajim.p.syntax_highlight')
class ChatSyntaxHighlighter:
def hide_code_markup(self, buf, start, end):
tag = buf.get_tag_table().lookup('hide_code_markup')
......@@ -27,8 +26,7 @@ class ChatSyntaxHighlighter:
line_break = self.config.get_line_break_action()
return (line_break == LineBreakOptions.ALWAYS) \
or (is_multiline and line_break == LineBreakOptions.MULTILINE)
or (is_multiline and line_break == LineBreakOptions.MULTILINE)
def format_code(self, buf, s_tag, s_code, e_tag, e_code, language):
style = self.config.get_style_name()
......@@ -37,26 +35,26 @@ class ChatSyntaxHighlighter:
self.hide_code_markup(buf, e_code, e_tag)
else:
comment_tag = GTKFormatter.create_tag_for_token(
pygments.token.Comment,
pygments.styles.get_style_by_name(style))
pygments.token.Comment,
pygments.styles.get_style_by_name(style))
buf.get_tag_table().add(comment_tag)
buf.apply_tag(comment_tag, s_tag, s_code)
buf.apply_tag(comment_tag, e_tag, e_code)
code = s_code.get_text(e_code)
log.debug("full text to encode: %s.", code)
log.debug('full text to encode: %s.', code)
start_mark = buf.create_mark(None, s_code, False)
start_mark = buf.create_mark(None, s_code, False)
lexer = None
if language is None:
lexer = self.config.get_default_lexer()
log.info("No Language specified. Falling back to default lexer: %s.",
self.config.get_default_lexer_name())
log.info('No Language specified. '
'Falling back to default lexer: %s.',
self.config.get_default_lexer_name())
else:
log.debug("Using lexer for %s.", str(language))
log.debug('Using lexer for %s.', str(language))
lexer = self.config.get_lexer_with_fallback(language)
if lexer is None:
......@@ -71,13 +69,15 @@ class ChatSyntaxHighlighter:
def find_multiline_matches(self, text):
start = None
matches = []
#Less strict, allow prefixed whitespaces: for i in re.finditer(r'(?:^|\n)[ |\t]*(```)\S*[ |\t]*(?:\n|$)', text, re.DOTALL):
# Less strict, allow prefixed whitespaces:
# for i in re.finditer(r'(?:^|\n)[ |\t]*(```)\S*[ |\t]*(?:\n|$)',
# text, re.DOTALL):
for i in re.finditer(r'(?:^|\n)(```)\S*(?:\n|$)', text, re.DOTALL):
if start is None:
start = i
elif re.match(r'^\n```', i.group(0)) is not None:
matches.append(
(start.start(), i.end(), text[start.start():i.end()]))
(start.start(), i.end(), text[start.start():i.end()]))
start = None
else:
# not an end...
......@@ -91,37 +91,40 @@ class ChatSyntaxHighlighter:
defined in XEP-0393.
The same applies mirrored to the end marker.
"""
return [(i.start(1), i.end(1), i.group(1)) for i in \
re.finditer(r'(?:^|\s|\*|~|_)(`((?!`).+?)`)(?:\s|\*|~|_|$)', text)]
return [(i.start(1), i.end(1), i.group(1)) for i in
re.finditer(r'(?:^|\s|\*|~|_)(`((?!`).+?)`)(?:\s|\*|~|_|$)',
text)]
def merge_match_groups(self, real_text, inline_matches, multiline_matches):
it_inline = iter(inline_matches)
it_multi = iter(multiline_matches)
length = len(real_text)
it_multi = iter(multiline_matches)
length = len(real_text)
# Just to get cleaner code below...
def get_next(iterator):
return next(iterator, (length, length, ""))
return next(iterator, (length, length, ''))
# In order to simplify the process, we use the 'length' here.
cur_inline = get_next(it_inline)
cur_multi = get_next(it_multi)
cur_multi = get_next(it_multi)
pos = 0
# This will contain tuples with parts of the input and its classification
parts = []
# This will contain tuples with parts of the input and its
# classification
parts = []
while pos < length:
log.debug("-> in: %s", str(cur_inline))
log.debug("-> mu: %s", str(cur_multi))
log.debug('-> in: %s', str(cur_inline))
log.debug('-> mu: %s', str(cur_multi))
# selected = (start, end, type)
selected = (cur_inline[0], cur_inline[1], MatchType.INLINE) \
if cur_inline[0] < cur_multi[0] \
else (cur_multi[0], cur_multi[1], MatchType.MULTILINE) \
if cur_multi[0] < length \
else (pos, length, MatchType.TEXT)
log.debug("--> select: %s", str(selected))
if cur_inline[0] < cur_multi[0]:
selected = (cur_inline[0], cur_inline[1], MatchType.INLINE)
elif cur_multi[0] < length:
selected = (cur_multi[0], cur_multi[1], MatchType.MULTILINE)
else:
selected = (pos, length, MatchType.TEXT)
log.debug('--> select: %s', str(selected))
# Handle plain text string parts (and unforseen errors...)
if pos < selected[0]:
......@@ -129,7 +132,7 @@ class ChatSyntaxHighlighter:
parts.append((real_text[pos:end], MatchType.TEXT))
pos = selected[0]
elif pos > selected[0]:
log.error("Should not happen, position > found match.")
log.error('Should not happen, position > found match.')
# Cut out and append selected text segment
parts.append((real_text[selected[0]:selected[1]], selected[2]))
......@@ -158,67 +161,76 @@ class ChatSyntaxHighlighter:
fixed = (marker_len_no_newline + 1, '\n')
return fixed
buf = self.textview.tv.get_buffer()
# first, try to find inline or multiline code snippets
inline_matches = self.find_inline_matches(real_text)
multiline_matches = self.find_multiline_matches(real_text)
# First, try to find inline or multiline code snippets
inline_matches = self.find_inline_matches(real_text)
multiline_matches = self.find_multiline_matches(real_text)
if not inline_matches and not multiline_matches:
log.debug("Stopping early, since there is no code block in it....")
log.debug('Stopping early, since there is no code block in it...')
return
iterator = iter_ if iter_ is not None else buf.get_end_iter()
iterator = iter_ if iter_ is not None else buf.get_end_iter()
# Create a start marker with left gravity before inserting text.
start_mark = buf.create_mark("SHP_start", iterator, True)
end_mark = buf.create_mark("SHP_end", iterator, False)
start_mark = buf.create_mark('SHP_start', iterator, True)
end_mark = buf.create_mark('SHP_end', iterator, False)
insert_newline_for_multiline = self.check_line_break(True)
insert_newline_for_inline = self.check_line_break(False)
insert_newline_for_multiline = self.check_line_break(True)
insert_newline_for_inline = self.check_line_break(False)
split_text = self.merge_match_groups(
real_text, inline_matches, multiline_matches)
real_text, inline_matches, multiline_matches)
buf.begin_user_action()
for num, (text_to_insert, match_type) in enumerate(split_text):
language = None
end_of_message = num == (len(split_text) - 1)
language = None
end_of_message = num == (len(split_text) - 1)
if match_type == MatchType.TEXT:
self.textview.detect_and_print_special_text(
text_to_insert, other_tags, graphics=_graphics,
iter_=iterator, additional_data=_additional)
text_to_insert, other_tags, graphics=_graphics,
iter_=iterator, additional_data=_additional)
else:
if match_type == MatchType.MULTILINE:
language_match = re.search(
'\n*```([^\n]*)\n', text_to_insert, re.DOTALL)
'\n*```([^\n]*)\n', text_to_insert, re.DOTALL)
language = None if language_match is None \
else language_match.group(1)
else language_match.group(1)
language_len = 0 if language is None else len(language)
# We account the language word width for the front marker
front = fix_newline(text_to_insert[0], 3 + language_len,
insert_newline_for_multiline)
back = fix_newline(text_to_insert[-1], 3,
insert_newline_for_multiline and not end_of_message)
front = fix_newline(
text_to_insert[0],
3 + language_len,
insert_newline_for_multiline)
back = fix_newline(
text_to_insert[-1],
3,
insert_newline_for_multiline and not end_of_message)
else:
front = fix_newline(text_to_insert[0], 1,
insert_newline_for_inline)
back = fix_newline(text_to_insert[-1], 1,
insert_newline_for_inline and not end_of_message)
front = fix_newline(
text_to_insert[0],
1,
insert_newline_for_inline)
back = fix_newline(
text_to_insert[-1],
1,
insert_newline_for_inline and not end_of_message)
marker_widths = (front[0], back[0])
text_to_insert = ''.join([front[1], text_to_insert, back[1]])
# insertion invalidates iterator, let's use our start mark...
# Insertion invalidates iterator, let's use our start mark...
self.insert_and_format_code(buf, text_to_insert, language,
marker_widths, start_mark, end_mark, other_tags)
iterator = buf.get_iter_at_mark(end_mark)
# the current end of the buffer's contents is the start for the
# The current end of the buffer's contents is the start for the
# next iteration
buf.move_mark(start_mark, iterator)
......@@ -235,7 +247,7 @@ class ChatSyntaxHighlighter:
def insert_and_format_code(self, buf, insert_text, language, marker,
start_mark, end_mark, other_tags=None):
start_iter = buf.get_iter_at_mark(start_mark)
start_iter = buf.get_iter_at_mark(start_mark)
if other_tags:
buf.insert_with_tags_by_name(start_iter, insert_text,
......@@ -243,14 +255,14 @@ class ChatSyntaxHighlighter:
else:
buf.insert(start_iter, insert_text)
tag_start = buf.get_iter_at_mark(start_mark)
tag_end = buf.get_iter_at_mark(end_mark)
s_code = tag_start.copy()
e_code = tag_end.copy()
tag_start = buf.get_iter_at_mark(start_mark)
tag_end = buf.get_iter_at_mark(end_mark)
s_code = tag_start.copy()
e_code = tag_end.copy()
s_code.forward_chars(marker[0])
e_code.backward_chars(marker[1])
log.debug("full text between tags: %s.", tag_start.get_text(tag_end))
log.debug('full text between tags: %s.', tag_start.get_text(tag_end))
self.format_code(buf, tag_start, s_code, tag_end, e_code, language)
......@@ -266,6 +278,6 @@ class ChatSyntaxHighlighter:
buf.apply_tag(tag, tag_start, tag_end)
def __init__(self, config, textview):
self.last_end_mark = None
self.config = config
self.textview = textview
self.last_end_mark = None
self.config = config
self.textview = textview
syntax_highlight/config_dialog.ui
View file @ c60deaea
This diff is collapsed.
syntax_highlight/gtkformatter.py
View file @ c60deaea
import logging
from gi.repository import Gtk as gtk
from gi.repository import Gtk
from gi.repository import Pango
from pygments.formatter import Formatter
from gajim.plugins.helpers import log
log = logging.getLogger('gajim.p.syntax_highlight')
class GTKFormatter(Formatter):
name = 'GTK Formatter'
aliases = ['textbuffer', 'gtk']
......@@ -14,14 +15,14 @@ class GTKFormatter(Formatter):
def __init__(self, **options):
super(GTKFormatter, self).__init__(**options)
#Formatter.__init__(self, **options)
# Formatter.__init__(self, **options)
self.tags = {}
self.mark = options.get('start_mark', None)
@staticmethod
def create_tag_for_token(ttype, highlighting_style):
style = highlighting_style.style_for_token(ttype)
tag = gtk.TextTag.new()
tag = Gtk.TextTag.new()
if 'bgcolor' in style and not style['bgcolor'] is None:
tag.set_property('background', '#%s' % style['bgcolor'])
if 'bold' in style and style['bold']:
......@@ -44,13 +45,13 @@ class GTKFormatter(Formatter):
tag.set_property('underline', 'single')
return tag
def get_tag(self, ttype, buf):
"""
Creates, stores and returs a tag for a given token type.
This method ensures that a tag style is created only once.
Furthermore, the tag will be added to the given Gtk.TextBuffer's tag table.
Furthermore, the tag will be added to the given Gtk.TextBuffer's
tag table.
"""
tag = None
if ttype in self.tags:
......@@ -65,23 +66,23 @@ class GTKFormatter(Formatter):
self.mark = mark
def format(self, tokensource, outfile):
if not isinstance(outfile, gtk.TextBuffer) or outfile is None:
if not isinstance(outfile, Gtk.TextBuffer) or outfile is None:
log.warn("Did not get a buffer to format...")
return
buf = outfile
end_iter = buf.get_end_iter()
end_iter = buf.get_end_iter()
start_mark = self.mark
start_iter = buf.get_iter_at_mark(start_mark) if not start_mark is None \
else end_iter
start_mark = self.mark
start_iter = buf.get_iter_at_mark(start_mark) if \
start_mark is not None else end_iter
last_ttype = None
last_start = start_iter
last_end = buf.get_end_iter()
last_ttype = None
last_start = start_iter
last_end = buf.get_end_iter()
last_fixed_start = last_start
reset = True
reset = True
for ttype, value in tokensource:
search = None
......@@ -90,24 +91,26 @@ class GTKFormatter(Formatter):
buf.apply_tag(tag, last_fixed_start, last_end)
search = last_end.forward_search(value, gtk.TextSearchFlags.TEXT_ONLY, end_iter)
search = last_end.forward_search(
value, Gtk.TextSearchFlags.TEXT_ONLY, end_iter)
reset = True
else:
# in case last_ttype is None, i.e. first loop walkthrough:
# In case last_ttype is None, i.e. first loop walkthrough:
# last_start to end_iter is the full code block.
search = last_start.forward_search(value, gtk.TextSearchFlags.TEXT_ONLY, end_iter)
search = last_start.forward_search(
value, Gtk.TextSearchFlags.TEXT_ONLY, end_iter)
# Prepare for next iteration
last_ttype = ttype
if search is not None:
(last_start, last_end) = search
# If we've found the end of a sequence of similar type tokens or if
# we are in the first loop iteration, set the fixed point
# If we've found the end of a sequence of similar type tokens
# or if we are in the first loop iteration, set the fixed point
if reset:
last_fixed_start = last_start
reset = False
else:
# Hm... Nothing found, but tags left? Seams there's nothing we
# Hm... Nothing found, but tags left? Seems there's nothing we
# can do now.
break
syntax_highlight/plugin_config.py
View file @ c60deaea
from gajim.plugins.helpers import log
import logging
from gi.repository import Gdk
from pygments.lexers import get_lexer_by_name, get_all_lexers
from pygments.lexers import get_lexer_by_name
from pygments.lexers import get_all_lexers
from pygments.styles import get_all_styles
from pygments.util import ClassNotFound
from syntax_highlight.types import LineBreakOptions
from syntax_highlight.types import CodeMarkerOptions
from syntax_highlight.types import PLUGIN_INTERNAL_NONE_LEXER_ID
log = logging.getLogger('gajim.p.syntax_highlight')
from .types import LineBreakOptions, CodeMarkerOptions, \
PLUGIN_INTERNAL_NONE_LEXER_ID
class SyntaxHighlighterConfig:
PLUGIN_INTERNAL_NONE_LEXER=('None (monospace only)', PLUGIN_INTERNAL_NONE_LEXER_ID)
PLUGIN_INTERNAL_NONE_LEXER = ('None (monospace only)',
PLUGIN_INTERNAL_NONE_LEXER_ID)
def _create_lexer_list(self):
# The list we create here contains the plain text name and the lexer's
# id string
lexers = []
# Iteration over get_all_lexers() seems to be broken somehow. Workarround
# Iteration over get_all_lexers() seems to be broken somehow
# Workaround
all_lexers = get_all_lexers()
for lexer in all_lexers:
# We don't want to add lexers that we cant identify by name later
......@@ -24,7 +32,7 @@ class SyntaxHighlighterConfig:
lexers.append((lexer[0], lexer[1][0]))
lexers.sort()
# Insert our internal "none" type at top of the list.
# Insert our internal 'none' type at top of the list
lexers.insert(0, self.PLUGIN_INTERNAL_NONE_LEXER)
return lexers
......@@ -38,24 +46,24 @@ class SyntaxHighlighterConfig:
lexer = None
try:
lexer = get_lexer_by_name(name)
except:
except ClassNotFound:
pass
return lexer
def get_lexer_with_fallback(self, language):
lexer = self.get_lexer_by_name(language)
if lexer is None:
log.info("Falling back to default lexer for %s.",
self.get_default_lexer_name())
log.info('Falling back to default lexer for %s.',
self.get_default_lexer_name())
lexer = self.default_lexer[1]
return lexer
def set_font(self, font):
if font is not None and font != "":
if font is not None and font != '':
self.config['font'] = font
def set_style(self, style):
if style is not None and style != "":
if style is not None and style != '':
self.config['style'] = style
def set_line_break_action(self, option):
......@@ -68,14 +76,16 @@ class SyntaxHighlighterConfig:
lexer = get_lexer_by_name(name)
if lexer is None and self.default_lexer is None:
log.error("Failed to get default lexer by name."\
"Falling back to simply using the first in the list.")
log.error('Failed to get default lexer by name.'
'Falling back to simply using the first lexer '
'in the list.')
lexer = self.lexer_list[0]
name = lexer[0]
name = lexer[0]
self.default_lexer = (name, lexer)
if lexer is None and self.default_lexer is not None:
log.info("Failed to get default lexer by name, keeping previous"\
"setting (lexer = %s).", self.default_lexer[0])
log.info('Failed to get default lexer by name, keeping '
'previous setting (lexer = %s).',
self.default_lexer[0])
name = self.default_lexer[0]
else:
self.default_lexer = (name, lexer)
......@@ -88,7 +98,7 @@ class SyntaxHighlighterConfig:
self.config['bgcolor_override'] = state
def set_bgcolor(self, color):
if isinstance(color, Gdk.Color):
if isinstance(color, Gdk.RGBA):
color = color.to_string()
self.config['bgcolor'] = color
......@@ -110,9 +120,9 @@ class SyntaxHighlighterConfig:
return self.lexer_list
def get_line_break_action(self):
# return int only
# Return int only
if isinstance(self.config['line_break'], int):
# in case of legacy settings, convert.
# In case of legacy settings, convert.
action = self.config['line_break']
self.set_line_break_action(action)
else:
......@@ -146,13 +156,13 @@ class SyntaxHighlighterConfig:
Initialize all config variables that depend directly on pygments being
available.
"""
self.lexer_list = self._create_lexer_list()
self.style_list = [s for s in get_all_styles()]
self.lexer_list = self._create_lexer_list()
self.style_list = [s for s in get_all_styles()]