Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Python - for loop not appending array with regex [SOLVED]
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6656
Location: The soundosphere

PostPosted: Mon Jul 22, 2024 12:48 am    Post subject: Python - for loop not appending array with regex [SOLVED] Reply with quote

The following code fetches version numbers from the Realtime Kernel project, and then for each version number, goes to the page for that version number and fills an array with a specific pattern for a filename. The resulting array should contain a list of file names for each version number, but it appears to only contain early versions (2.6). Using print statements, I can see that the code works in that it fetches the sha256sums.asc files - all of them, all versions. I'm guessing I don't yet understanding something about populating Python arrays, and that my code isn't allowing the patches_full_versions array to include everything from versions 2.6 to 6.9, as expected.

If the array was somehow getting reset, I'd expect it to contain only the latest version number, but the opposite is happening - it only contains the earliest. It's like it simply stopped there even though the code is continuing to fetch info for later versions.

Code:

    patches_versions = []
    patches_full_versions = []

    RT_PATCHES_BASE_URL = "https://cdn.kernel.org/pub/linux/kernel/projects/rt/"

    cleaner = re.compile("<.*?>") # For removing HTML tags later.

    # Create array of patches versions:
    patches_page_content = requests.get(RT_PATCHES_BASE_URL)
    patches_page_content.raise_for_status()
    stripped_content = re.sub(cleaner, "", patches_page_content.text)
    for line in stripped_content.splitlines():
        x = False
        x = re.findall(r"[0-9]", line)
        if not line == "":
            if x:
                patches_versions.append(line.split("/")[0])

    patch_name_pattern = re.compile(r'patch-.*?\.tar\.xz')
    for x in patches_versions:
        patch_version_page_content = requests.get(f"{RT_PATCHES_BASE_URL}{x}/sha256sums.asc")
        patch_version_page_content.raise_for_status()
        for match in re.findall(patch_name_pattern, patch_version_page_content.text):
            patches_full_versions.append(match)

_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN


Last edited by audiodef on Mon Jul 22, 2024 5:27 pm; edited 1 time in total
Back to top
View user's profile Send private message
eschwartz
Developer
Developer


Joined: 29 Oct 2023
Posts: 131

PostPosted: Mon Jul 22, 2024 3:29 am    Post subject: Reply with quote

It's a data matching problem. Consider the following debug technique:

Code:

>>> for x in patches_versions:
...     patch_version_page_content = requests.get(f"{RT_PATCHES_BASE_URL}{x}/sha256sums.asc")
...     patch_version_page_content.raise_for_status()
...     m = [match for match in re.findall(patch_name_pattern, patch_version_page_content.text)]
...     print(f'---------> discovered versions: {m}')
...     print(patch_version_page_content.text)
...


We see for example the following match:
Code:

---------> discovered versions: ['patch-2.6.29.6-rt24-broken-out.tar.xz']
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

9f25f4a0deddacf9199b85ae3ffdb605602c8ddf3d362aae92112a8735cbaf9d  patch-2.6.29.6-rt24
39118b6159f5eb188420adce67221bef50c8680774dd0b9461d532976973d50b  patch-2.6.29.6-rt24-broken-out.tar
a4dcb4798d69e7d37e1150bd85375079dab024aaf807d746ac1c3287a9b32133  patch-2.6.29.6-rt24-broken-out.tar.bz2
cbc987982654d183001ee29d6c69531690f370f801def022b0a3c8fcdad15ce6  patch-2.6.29.6-rt24-broken-out.tar.gz
96956df74ca8845998de9a9112be6b7d2e90ff8169dc887439410099b7899f69  patch-2.6.29.6-rt24-broken-out.tar.xz
616aabdc2c76d0edf283a13567382f5a18ddaae6ec7beab7dc88de30e4b4794f  patch-2.6.29.6-rt24.bz2
564d5ee546ff5f22fd7ce868f76807f3b00052c72c586d421bb01cb2e0c1d2c8  patch-2.6.29.6-rt24.gz
0d189a13d5c3a6a8e9cfe713ba026ee4ff47b69515616e26e694e1aa67fe2b44  patch-2.6.29.6-rt24.xz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBCAAGBQJkSEJXAAoJEGMtOgZYnaax37MP+wX4pH7D1p5DQgxUws740nYN
GH9IqFq83hhEWQx4O5+PVhUKUTgMcEyraEv6ghXDczowDNuS+DlPVGIa/4uJdwL6
srNEnyYmc82aMlU/Bv6BAbbRPtcqZ/hbc47D4DZ6f/5nTzaMhLnx803jeB9QPuuF
D1e9JMH3Tfow0rGXbQtBLDSQIoQd1qvDShKtCh1WNc9zd+iSVQhfIe5pXlPlug0A
9JrobGv0ULSOju20I4wukllc/xAFB3HwpPUQ+z4QHsOOEt/V4hiZg4PEicFtXLUd
cq1kHjmF5Lcbu3Ob0+TR5E0wznyCMkhYXECOK8rTTLrrRAJhBWyffDmHBCtJJ0Iu
IeyZBhVf8QTnZ4d+reZkQGqYEjhJnOdgIAx3X0YL6qDZliwHus4B0oibQY0+Wxnt
IA0BCrG2A3ZTZCl56e/0AHn2gdpn3ZbT68ZOnJaP3MG5T8R4EBWRK3mHi9zUi0JZ
gJ914hu2UDZrXQ7eOIoMl8/exLBBolwBtVTIw+LR/PYf3Es0wyxbdxEwdMGno4r+
1/cj3SkoUhORFDDLs1dvcspIyaLHATfKTJMbkexSGTKi3aA1BZ6MOixDKyDbKAmY
ijBa2kifawIuVXLI/IDpUauZ+4GBYPQWR9SpmuY6ik1SxODPqK2BoOakfcJCSfiC
cP34TGsSxUkbeADge1L+
=o2uI
-----END PGP SIGNATURE-----



And then we see the next one fail:
Code:

---------> discovered versions: []
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

28c0ada5fa297f9d24870fd030363721c70b3b1e4601b731f1471259dd472fe6  patch-2.6.31.12-rt21.patch.gz
1e096eb8cafe32d1674c519d85a90bc837930221b54bca476ffc5bec72f9ce3c  patch-2.6.31.12-rt21.patch.xz
e1cd88349a3f92f5d7181b866ef69e7b37086013aee0a1073fbb5b7084c71432  patch-2.6.31.6-rt19
0c458a77e763d03691fe8e5afdee239dff2975614275b614bd2be7aceffdfa44  patch-2.6.31.6-rt19.bz2
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBCAAGBQJkSEJYAAoJEGMtOgZYnaaxxW4P/j1303gS8soqpY4pBpvw3wZq
iNI3q6hDirQ6H1b7h3z5wc6CdhRbYlU4wja8AsUomINIshF8g0Z8tFBSV2i2C+8k
mF2eovSL844i0IS01FfZop3xNasLJadf+/gepnxFQZNgOoBIIrFDAMC9Z3M+TbSh
gYzvfdK7ScZXno/kKMFs2PEYXq18W2oYZxfTbTFUFaEPgClTmg1JBH6mAVLRQBwv
HBWmBAILRlaxCuSPdC5L0dBSRRNZNNsQ8y6yEnuvLWiqS/ady+yjK0pNqMwmPoce
ZWXnBOA4ghyr+NEW5pZ4CM+D04oX+24sNZnlAHW6xjSVrQp8h555mVO1Jm/w6pQd
5YGrnWknGAaWmjKiV0lW+pzJ35kQl8WNEw8XR7N0Az3VgZAd4zjp2NvI8xgrAMLh
2vrkyOs6N7DmXr/r+USXcZXWtUyOrq+zll/mBNGggKD0+Ox7nmKqRGyGU4SKOZQ3
akI6LPn6opKamCTpEZzb6VckAOfc4Tb/gHvd1aHzxygEzlfPH3v2uFGeLEuOFR8r
8u99iFEXDF7CQU6Qw04yEJALlHLCM0wkL5XDpKp8gTm1e1lGk1EwuasTcnAFv41X
ImUhVnQ4A3gHKbm2CSuDegGFECXxmIt7oiNWKdD02u/S+pkcdR2r6BINlLvu3BYV
ajnvcCOEvj4qQnjCtbhl
=s3Rk
-----END PGP SIGNATURE-----



What's different about this one? It has no "broken-out.tar", just a patch.xz

The next one is even more fun:
Code:

---------> discovered versions: []
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

77ece8fbb843c187affab1f6102d6932057d5178144442512dd0321034fad2aa  patch-2.6.33.9-rt31
7855f812e6deacd5f875f1917699be2d58253db8e250f2b1975235806cb4c81f  patch-2.6.33.9-rt31.bz2
49ec98e95f84c61d33e9f4da12726407d2212948d9b641bc56b0022c63604a59  patch-2.6.33.9-rt31.gz
a3966c56606cb83b01c7a9dba88a97c8a18cc3cacb7ad3a48827ddc4fdf1abe0  patch-2.6.33.9-rt31.xz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBCAAGBQJkSEJYAAoJEGMtOgZYnaaxtBMP/22RPHesnXXUmtMV6kUzBMLW
55GrJz5JI71Ai5DjLTOdpL6tFucoGr3vHccc20oA1V1sVATUBEzGgOmAmd1MfRWq
WazvBwZaqox5ORFfElYXtbFUAb79nJ7QbAlIpqhIfJXLlP34GhkYb6v+LupctMet
Tpgvq+GqG1L/yd8VMH+lJ1kxuBwLZnwUvwN+ZIXeh9E0vT5Cwu1ZO5ppDG7INJ88
YT1x4H5sF58DJI0mI4Tq6AT7ovIjvHRidn+qen11OUdkX9+BU9PRA4rOgpZSmORV
YgzcmJFyep7K4RBRDZsPKimJ+AuZG6ma5kgmknOq2x8lDrIO/po0L9pw1LUKoCC3
ztuv9Mnv4vQ7urE1X4E1xVziaA/vEdO8td5NYL04YS62oJiybl9TVnrNfnFGAdeR
WUvQ/mvHIOXcF7amUJxqz2s3codwVXdnDP7mWoull28UhUYm9wTrhXZCiTg4A3MA
5qPgwmHIhSlKtkAp0ZYFrq53bCXzHBp4UzZ8j4XYxl1Sa+SlGOzNHDYmHMy369vR
yWDFkDyI8VM6M65DfAN3zxx/RyulKr2GU895Sv3pfKKHhTQbkROHTwIApd7FW+W+
T98JgmbaIB2k+ONzMSCqKXvcbBlU3DqVc+iExKLuTFi2JkehYA1NgZ7NkwkTrwgP
GT0X5qZBux06mrX/D9eO
=yY/T
-----END PGP SIGNATURE-----



Still no results. This time, it's just "xz".

Past here, things become more regular:
Code:

---------> discovered versions: []
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

3e88a5ce24ae7747f1d3962d550a784c5dbab80ca7519e5a22fa3c57dae72694  README
8dc0fe9e0b0498d943cae7d4e46acaa57ee2c33c7a97a57eab6c8110025ebc12  patch-3.0.101-rt130.patch.bz2
fb0b38a3d6203d1364cd6fd0c328415178bccba6f42ba9aded1ba952729f5826  patch-3.0.101-rt130.patch.gz
e9dbf1bf29e50b73aa48aea4efccdf4aaffff0f7010667650c09d1a19a8ec050  patch-3.0.101-rt130.patch.xz
fa985df338b83425bb280d20ecae236e3377c26380a7818be499ec5f04e51aec  patches-3.0.101-rt130.tar.bz2
dea07fc89d3613b3e1206106819bd5131f9672a11d93fb4b259f743ff03e58f2  patches-3.0.101-rt130.tar.gz
4ac22daed9b26fc6ce4069ce2924131a2e95c2490ac04e425c74203eabd82102  patches-3.0.101-rt130.tar.xz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBCAAGBQJkSEJZAAoJEGMtOgZYnaaxi2QQAJjK5z9c6GJ+rHv9m5d3DAJq
lVkPqwvorOYU2hpyn8mmyBitAC4lvoBi9RO+Kbg31ZYyPAk9PBfXoe2y6dmgGibN
0xyMfyyrNVUEdnDG5f7GxoS4fJ+oxHtbL9ZuQfC2FDSYij7Rm9HBsY27B4/N+GxZ
ZNji1qSpqqv7NDU6ifjvrwidpWIxNlh90NAx6adkKy3uTjUTH0NVgMll0CKD/D/C
XuF1MvkeNLfoTJUkc/LWk5YDJofjL/ktC38iy+5DEhR5WZDmtHcd6SIiGMzBmCON
cHd6NyBOBntFRe39cM1fbmoqLsEBtzZM7Q3IE4w4nkCDzSueypfFWWMHq08sR12v
tUy4NieW5G3gpSaxpqsmB6NlgCeQTkO71I7wEGLJY8uBhwXjbg5XhAr+OMWM5dag
ubiWwk4zoRuiDr55VeN83x5F9iB60AyJndd8I+Vfjy0bVVcplQcDdf376hM0VG6T
Vr+ODxidYkmtPomdmIWrzdDtOPRkJ8EXLt5F+tu1Pwq7rymre2J2sgV6+9C+2qt/
BVAlt9As8lg19Yoq1ViKZ/KNaDKji2NrG2GhfqUG+PzN+9S5k6Z4eL7wEcHxwBMc
RupJK2PDC3Go+1ApBL98B7gB88hY+y5m0XDQBJHHAErNtgciJC56QJ24yrBpp+Aq
wrm0qDUyMTvJ1yRSOOYL
=gwHr
-----END PGP SIGNATURE-----



There is:
- patch-.*?\.patch\.xz
- patches-.*?\.tar\.xz

and your original match fails because of the "es" in "patches".
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6656
Location: The soundosphere

PostPosted: Mon Jul 22, 2024 5:27 pm    Post subject: Reply with quote

Yep. Classic me - simple typo, cannot see it to save my life. Much obliged, eschwartz.
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum