Repository navigation

AOP-wiki as a new database + text categorization - #136

Merged

hechth merged 186 commits into

mainfrom

rdurnik/issue117

Sep 23, 2026

Commits on May 20, 2026

get abstracts from aop-wiki
rdurnik
committed
0a3a101
prompt to categorize text
rdurnik
committed
4ccfb7b
added sparqlwrapper
rdurnik
committed
09eeb38
fixed dep
rdurnik
committed
1501095

Commits on Jun 17, 2026

Update README.md by removing old badges
rdurnik
committed
7687424
badges update
rdurnik
committed
0867fb2
badges update
rdurnik
committed
b366e29
badges update
rdurnik
committed
4b000d6
badges update
rdurnik
committed
77249da
badges update
rdurnik
committed
fd5c7b3
update badges
rdurnik
committed
8d3755d
Badges update
rdurnik
committed
0a1cb32
Add funding section to README
rdurnik
committed
0b61160
Reorganize Funding section in README.md
rdurnik
committed
688c189
created dockerfile
rdurnik
committed
2adab8b
created .json file
rdurnik
committed
2ffa0c4
updated readme
rdurnik
committed
57e0c98
added dockerignore
rdurnik
committed
0f79654
removed enfile
rdurnik
committed
12a7f33
introduced ids as parameters for download

hechth
authored and
rdurnik
committed
ba6d02f
starting update on PMC

hechth
authored and
rdurnik
committed
f5f5a38
removed pagination
rdurnik
committed
a5192f3
europepmc text fix - get_ids is independent
rdurnik
committed
19d7af3
keeping query empty to avoid confussion
rdurnik
committed
95e6a9f
europepmc mock test updates
rdurnik
committed
2401b86
lint
rdurnik
committed
4577d90
pmc tests fix
rdurnik
committed
9b58871
pubmed ids independent
rdurnik
committed
0782b19
lint
rdurnik
committed
df12dbc
mypy fixes
rdurnik
committed
aa2d95f
lint
rdurnik
committed
48e8367
added type checke to ci
rdurnik
committed
6cf9da9
lint
rdurnik
committed
c48841c
removed tests
rdurnik
committed
7411708
ignore type: ignore[assignment] for api keys
rdurnik
committed
2373dfe
lint
rdurnik
committed
655288a
re-add mypy for tests to CI
rdurnik
committed
49120e2
mypy for tests fix
rdurnik
committed
db0826e
lint
rdurnik
committed
90814ba
fix pmc could return a path even if download silently fails
rdurnik
committed
eac30ca
if response.ok replaced with raise_for_status() for europe pmc
rdurnik
committed
6f2033e
added option to disable image download
rdurnik
committed
5900585
lint
rdurnik
committed
66cc910
added test for empty publications europe pmc
rdurnik
committed
9cf7b34
images are converted to one format
rdurnik
committed
b683790
added PIL
rdurnik
committed
31b4d32
fixed pil to pillow
rdurnik
committed
99a46ed
fixed util on ci
rdurnik
committed
2445399
added sorted
rdurnik
committed
f333e1a
allow for size difference for windows
rdurnik
committed
a5f30a1
added rel for utils
rdurnik
committed
84480f5
added option to set up the unified image format
rdurnik
committed
72bb9cb
moved relationship_type to proper folder
rdurnik
committed
df9746c
lint
rdurnik
committed
a978ac6
added markxfail
rdurnik
committed
f9ef018
removed get_ids from pmc constructor
rdurnik
committed
d15ff0b
removed get_ids from pubmed constructor
rdurnik
committed
feb176e
fixed pubmed mock tests
rdurnik
committed
4d366de
query datastructure draft
rdurnik
committed
06f8ee2
created and implemented query object
rdurnik
committed
cd0614d
added query for pubmed
rdurnik
committed
4671bbd
fixed europepmc tests being ran multiple times
rdurnik
committed
c06f277
removed filtering for abs/title to avoid confusion and added option to remove preprints
rdurnik
committed
e69e694
updated options for pmc
rdurnik
committed
2eb8746
added tests for pmc
rdurnik
committed
a204ed5
changed date to str as the format must include 0 at the start if there is only one numerical value for day/month
rdurnik
committed
132815e
updated to str for date in tests
rdurnik
committed
a65f221
europepmc filtering fix
rdurnik
committed
c29fefb
added test for europe pmc filtering
rdurnik
committed
635fc4b
pubmed filtering update
rdurnik
committed
a436c3c
pubmed filter update
rdurnik
committed
af9c1d0
added tests for pubmed filtering
rdurnik
committed
7f609b2
changed descs
rdurnik
committed
e4e973e
removed unused _get_abstract from pubmed
rdurnik
committed
8cf6b7c
fixed get metadata to match get abstracts
rdurnik
committed
5213f80
lint
rdurnik
committed
fd56b43
moved async get ids to helper class
rdurnik
committed
db08633
made ncbi less dependent on pmc
rdurnik
committed
f5f8859
ncbi implements get id interface
rdurnik
committed
99190bc
pubmed is using ncbi get id
rdurnik
committed
ccf29e7
removed limit for pubmed
rdurnik
committed
f473a12
removed maximum_results var
rdurnik
committed
7bcf7d1
shared get_abstract in ncbi
rdurnik
committed
49732af
publication metadata moved to ncbi
rdurnik
committed
25cbfa3
removed leftover
rdurnik
committed
6e9dc23
removed search term from ncbi constructor
rdurnik
committed
e7efa60
prep for get abstract for pmc
rdurnik
committed
ad3f673
fix pubmed mock tests
rdurnik
committed
5f5cb9d
added test for multiple abstracts pubmed
rdurnik
committed
933ab5d
tests for pmc get abstracts
rdurnik
committed
c606c7d
pmc get abstract
rdurnik
committed
0213dea
lint
rdurnik
committed
d231e13
mypy fix
rdurnik
committed
dbaa69c
lint
rdurnik
committed
9269cf6
accounted for missing abstracts in pmc
rdurnik
committed
e6117b2
increased retries
rdurnik
committed
ee27043
moved semaphore and limiter to constructor
rdurnik
committed
c2ee151
added max retries fix
rdurnik
committed
4670763
fixed entrez retries
rdurnik
committed
375c3c8
prep for metadata extension
rdurnik
committed
fac19c7
improved for loop
rdurnik
committed
0535913
removed comment
rdurnik
committed
349d501
lint
rdurnik
committed
7b8832f
initial version of get metadata for pmc
rdurnik
committed
f327302
mypy fix
rdurnik
committed
10053ab
test files clean up
rdurnik
committed
4864c32
moved pmc prefix trimming to utils
rdurnik
committed
5207b4d
updated ratio
rdurnik
committed
1ee75ab
updated entrez retry
rdurnik
committed
3b7c39e
changed base image test file
rdurnik
committed
0c129dc
changed file name
rdurnik
committed
9c1cb59
fixed test
rdurnik
committed
4edbc7b
updated metadata for pmc
rdurnik
committed
af1c6b2
avoid returning empty metadata
rdurnik
committed
9a48951
added test for id
rdurnik
committed
70eaafa
updated metadata for pubmed
rdurnik
committed
caecba4
europe pmc metadata update
rdurnik
committed
56028ec
mypy fix
rdurnik
committed
ef95307
renamed test-data to test_data
rdurnik
committed
114a1c6
moved test data to the same folder
rdurnik
committed
007b190
changed desc id
rdurnik
committed
58a419d
changed name publication_metadata to metadata
rdurnik
committed
0b17a17
lint
rdurnik
committed
6ba4e69
lint
rdurnik
committed
6da3cb0
added eq for metadata
rdurnik
committed
a17b7f2
simplified hashing
rdurnik
committed
07ecdef
mypy fix
rdurnik
committed
1972836
reduced image size
rdurnik
committed
685743c
added parametrization to image conversion test
rdurnik
committed
cc7def0
More image conversion test cases. Note: .tif converted to .png has larger size than the other formats. Caused by RGB vs RGBA? I think this is OK because it's still converted to .png
rdurnik
committed
ec8cc7a
additional image format from europe pmc and pmc
rdurnik
committed
478f999
europe pmc get abstracts correctly returns empty list if none was found
rdurnik
committed
06fd269
increased ncbi sleep between retries
rdurnik
committed
bdb3a3a
lint
rdurnik
committed
71f59c0
readded get pdfs for europe pmc
rdurnik
committed
5b5b9d5
lint
rdurnik
committed
cb205d9
fix for empty year and empty authors metadata
rdurnik
committed
9992636
ncbi refactor
rdurnik
committed
f1815c5
pubmed general id is pmid
rdurnik
committed
ac4bbd0
metadata can be compared against id
rdurnik
committed
03e2265
implemented ncbi as private variable
rdurnik
committed
1314501
removed forgoten attributes pubmed
rdurnik
committed
26079d9
fixed ncbi desc
rdurnik
committed
8812986
extended examples

hechth
authored and
rdurnik
committed
432b099
slight refactor

hechth
authored and
rdurnik
committed
70dcf6b
limited error handling

hechth
authored and
rdurnik
committed
2107f8a
updated parsing logic and image conversions

hechth
authored and
rdurnik
committed
3442913
updated gitignore and merged local changes

hechth
authored and
rdurnik
committed
b339afe
updated example scripts

hechth
authored and
rdurnik
committed
1e217b0
allowed multiple relationship types and updated chemicals

hechth
authored and
rdurnik
committed
5bbfabc
removed .full_text from string in pmc
rdurnik
committed
c271dba
fixed europepmc tests wrong init
rdurnik
committed
22a98af
removed acetaminophen capital letter at the start
rdurnik
committed
d8099bb
fixed relationship types in tests
rdurnik
committed
7fb9611
test europepmc mypy fix
rdurnik
committed
44ba03e
lint
rdurnik
committed
1537952
removed europepmc commented out code for get_pdfs
rdurnik
committed
1763b7a
lint
rdurnik
committed
91746c2
examples lint
rdurnik
committed
f64bd5a
fixed examples
rdurnik
committed
d7ba933
renamed find_chemicals example to find_chemicals_relationships
rdurnik
committed
4faed05
lint
rdurnik
committed
53069d7
fixed network errors and added try except blocks europe pmc
rdurnik
committed
69c1eab
try except blocks pubmed and pmc
rdurnik
committed
3d4739e
mypy fix
rdurnik
committed
7288ae3
added saving text to files
rdurnik
committed
8b840ba
updated mock test pubmed
rdurnik
committed
8984162
updated examples
rdurnik
committed
a4cf022
lint
rdurnik
committed
b7a118c
lint
rdurnik
committed
0a91009
changed example to pmc
rdurnik
committed
c4e5b9e
added encoding to fix windows fail
rdurnik
committed
874eb32
bump version
rdurnik
committed
0cc7ce4
added sparqlwrapper
rdurnik
committed
192a8a8
updated prompt categorization
rdurnik
committed
a57121a
added interface
rdurnik
committed
8954632
added categorization text generation
rdurnik
committed
cb66b75
added tests
rdurnik
committed
1586ee1
added prompt specification for categorization
rdurnik
committed
028e204
updated categorization prompt
rdurnik
committed
7c9692b
modified get abstracts to not include empty abstracts
rdurnik
committed
3946938
updated prompt categorization
rdurnik
committed
33ed55d
added example for aop-wiki
rdurnik
committed
e4be8e4
Merge branch 'main' into rdurnik/issue117
rdurnik
authored
ef90703
lint
rdurnik
committed
34cb7c1
updated aop-wiki tests
rdurnik
committed
4a8423a