regex error : bad escape \w

[LeonLenclos]

Hello,

With Python 3.7 I have an error when the rive contain a _ (word wildcard).

The error is :
re.error: bad escape \w at position 16

in rivescript/brain.py l448 _ is replaced by a string containing \w

But python 3.7 does not want it :

Deprecated since version 3.5, will be removed in version 3.7: Unknown escapes in repl consisting of '' and an ASCII letter now raise a deprecation warning and will be forbidden in Python 3.7.
(source)

[brlala]

Have you found a workaround for this issue?

[kirsle]

It will probably need a code change in rivescript-python.

Is the exception raised on this line?

rivescript-python/rivescript/brain.py

Line 448 in b55c820

regexp = regexp.replace('_', '(\w+?)') # Convert _ into (\w+?)

regexp = regexp.replace('_', '(\w+?)')

It seems from the re docs, \w is still supported as a regexp metacharacter but its use is limited in functions like re.sub()... but this line is doing a string replace IIRC.

[brlala]

Hi @kirsle ! it's nice to see that you're still active on this project! This is what I found so far

1. The error only occurs in Python 3.7, works fine on Python 3.6
2. The rivescript entry causing the error has [_] which is being replaced with (\\w+?)
3. re.sub(pattern, repl, string, count=0, flags=0) the repl parameter now have unknown escapes reserved for future use.

Unknown escapes of ASCII letters are reserved for future use and treated as errors

Changed in version 3.6: Unknown escapes in pattern consisting of '' and an ASCII letter now are errors.
Changed in version 3.7: Unknown escapes in repl consisting of '' and an ASCII letter now are errors.
(source)

4. the following line in brain.py seems to be the cause due to the error mentioned above
   

   rivescript-python/rivescript/brain.py

   Lines 468 to 469 in b55c820

   	regexp = re.sub(r'\s*\[' + re.escape(match) + '\]\s*',
   	'(?:' + pipes + r'|(?:\\s|\\b))', regexp)

The values causing the error is the variable:
variable match = '(\\w+?)'
variable pipes = '(?:\\\\s|\\\\b)+(\\w+?)(?:\\\\s|\\\\b)+'

Trace log as follow:

Traceback (most recent call last):
  File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\sre_parse.py", line 1021, in parse_template
    this = chr(ESCAPES[this][1])
KeyError: '\\w'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Program Files\JetBrains\PyCharm 2018.3.4\helpers\pydev\_pydev_bundle\pydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "C:\Program Files\JetBrains\PyCharm 2018.3.4\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "C:/Users/User/PyCharmProjects/client/worker.py", line 21, in <module>
    import messaging_manager
  File "C:\Program Files\JetBrains\PyCharm 2018.3.4\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "C:\Users\User\PyCharmProjects\client\messaging_manager.py", line 10, in <module>
    import testing_manager as tm
  File "C:\Program Files\JetBrains\PyCharm 2018.3.4\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "C:\Users\User\PyCharmProjects\client\testing_manager.py", line 3, in <module>
    from chatbot_manager import handle_postbacks, handle_text_message
  File "C:\Program Files\JetBrains\PyCharm 2018.3.4\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "C:\Users\User\PyCharmProjects\client\chatbot_manager.py", line 71, in <module>
    rs.load_file("./rs/chitchat.rive")
  File "C:\Users\User\PyCharmProjects\client_venv\lib\site-packages\rivescript\rivescript.py", line 206, in load_file
    self._parse(filename, lines)
  File "C:\Users\User\PyCharmProjects\client_venv\lib\site-packages\rivescript\rivescript.py", line 265, in _parse
    self._precompile_regexp(trigger["trigger"])
  File "C:\Users\User\PyCharmProjects\client_venv\lib\site-packages\rivescript\rivescript.py", line 986, in _precompile_regexp
    self._regexc["trigger"][trigger] = self._brain.reply_regexp(None, trigger)
  File "C:\Users\User\PyCharmProjects\client_venv\lib\site-packages\rivescript\brain.py", line 469, in reply_regexp
    '(?:' + pipes + r'|(?:\\s|\\b))', regexp)
  File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\re.py", line 192, in sub
    return _compile(pattern, flags).sub(repl, string, count)
  File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\re.py", line 309, in _subx
    template = _compile_repl(template, pattern)
  File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\re.py", line 300, in _compile_repl
    return sre_parse.parse_template(repl, pattern)
  File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\sre_parse.py", line 1024, in parse_template
    raise s.error('bad escape %s' % this, len(this))
re.error: bad escape \w at position 16