Articles for tag: codecs

Search on blog:

Python: How to convert text with hex values and \ to normal characters

Sometimes you can scrape string with double \\ and hex values

\\x3Cstyle\\x3E\\x0A.mainDiv\\x0A\\x7B\\x0A\\x20\\x20width\\x3A1000px\\x3B\\x0A}\\x0A\\x3C/style\\x3E

But it should be

<style>
   .mainDiv
  {
  width:1000px;
  background-image:url
<style>

It needs to encode it back with raw_unicode_escape` and then decode with ``unicode_escape

text = """\\x3Cstyle\\x3E\\x0A.mainDiv\\x0A\\x7B\\x0A\\x20\\x20width\\x3A1000px\\x3B\\x0A}\\x0A\\x3C/style\\x3E"""

print(text)

text = text.encode('raw_unicode_escape').decode('unicode_escape')

print(text)

Notes:

Python doc: codecs / Python Specific Encodings / Text Encodings

Stackoverflow: How to Convert hex+character to Character in Python

Python: Jak zamienić tekst z wartosciami szestastwkoymi (hex) z \ na normalne znaki

Czasami podczas scrapingu można otrzymać tekst z podwójnymi \\ i szesnastkowymi kodami znaków

\\x3Cstyle\\x3E\\x0A.mainDiv\\x0A\\x7B\\x0A\\x20\\x20width\\x3A1000px\\x3B\\x0A}\\x0A\\x3C/style\\x3E

A to powinno wyglądać jako

<style>
   .mainDiv
  {
  width:1000px;
  background-image:url
<style>

Należy to przekonwerterować ponownie do bytes z użyciem raw_unicode_escape a …