scripts/multimodal/__pycache__/extract_meaning_kernels.cpython-312.pyc

<EFBFBD>
s<><73>iR?<00><00>T<00>dZddlZddlZddlZddlZddlmZddlmZddlm	Z	m
Z
mZmZddl
Z
	ddlmZdZ	ddlZdZ	ddlZdZGd<0C>d
<0A>ZGd<0E>d<0F>Zd<10>Zedk(re<19>yy#e$r
dZed	<09>Y<00>DwxYw#e$r
dZed
<EFBFBD>Y<00>RwxYw#e$r
dZed<0B>Y<00>`wxYw)z<>
Multimodal Meaning Kernel Extraction Pipeline
Extracts structured meaning kernels from academic PDF diagrams.
Issue #493: [Multimodal] Extract Meaning Kernels from Research Diagrams
<EFBFBD>N)<01>Path)<01>datetime)<04>List<73>Dict<63>Any<6E>Optional)<01>ImageTFz<Warning: PIL not available. Install with: pip install PillowzIWarning: pytesseract not available. Install with: pip install pytesseractzEWarning: pdf2image not available. Install with: pip install pdf2imagec
<00>h<00>eZdZdZ	d
dededededeeeff
d<08>Zd	efd
<EFBFBD>Z	d	eeeffd<0B>Z
d	efd<0C>Zy)<0E>
MeaningKernelz6Represents an extracted meaning kernel from a diagram.N<>	kernel_id<69>content<6E>source<63>
confidence<EFBFBD>metadatac<00><><00>||_||_||_||_|xsi|_tj<00>j<00>|_|j<00>|_
y<00>N)rr
rrrr<00>now<6F>	isoformat<61>	timestamp<6D>_generate_hash<73>hash)<06>selfrr
rrrs      <20>L/private/tmp/timmy-config-burn/scripts/multimodal/extract_meaning_kernels.py<70>__init__zMeaningKernel.__init__)sO<00><00>"<22><04><0E><1E><04><0C><1C><04><0B>$<24><04><0F> <20><0E>B<EFBFBD><04>
<0A>!<21><1C><1C><1E>1<>1<>3<><04><0E><18>'<27>'<27>)<29><04>	<09><00>returnc<00><><00>|j<00>d|j<00>d|j<00>d|j<00><00>}t	j
|j
<00><00>j<00>ddS)z'Generate a unique hash for this kernel.<2E>:N<>)rr
rr<00>hashlib<69>sha256<35>encode<64>	hexdigest)r<00>content_strs  rrzMeaningKernel._generate_hash3sX<00><00><1D><1E><1E>(<28><01>$<24>,<2C>,<2C><1E>q<EFBFBD><14><1B><1B>
<0A>Q<EFBFBD>t<EFBFBD>~<7E>~<7E>FV<46>W<><0B><16>~<7E>~<7E>k<EFBFBD>0<>0<>2<>3<>=<3D>=<3D>?<3F><03><12>D<>Drc<00><><00>|j|j|j|j|j|j
|jd<01>S)z(Convert to dictionary for serialization.<2E>rr
rrrrrr&<00>rs r<00>to_dictzMeaningKernel.to_dict8s?<00><00><1E><1E><1E><1B>|<7C>|<7C><1A>k<EFBFBD>k<EFBFBD><1E>/<2F>/<2F><1C>
<0A>
<0A><1D><1E><1E><18>I<EFBFBD>I<EFBFBD>
<EFBFBD>	
rc<00>B<00>d|j<00>d|jdd<00>d<04>S)NzKernel[z]: <20>d<00>...)rr
r's r<00>__str__zMeaningKernel.__str__Ds(<00><00><18><14><1E><1E>(<28><03>D<EFBFBD>L<EFBFBD>L<EFBFBD><14>#<23>,><3E>+?<3F>s<EFBFBD>C<>Cr)<02>N)<0C>__name__<5F>
__module__<EFBFBD>__qualname__<5F>__doc__<5F>str<74>floatrrrrr(r,<00>rrrr&sn<00><00>@<40>FJ<01>*<2A>#<23>*<2A><03>*<2A>S<EFBFBD>*<2A>"<22>*<2A>48<34><13>c<EFBFBD><18>N<EFBFBD>*<2A>E<01><03>E<01>


<EFBFBD><14>c<EFBFBD>3<EFBFBD>h<EFBFBD><1E>

<EFBFBD>D<01><13>Drrc<00>L<00>eZdZdZddeeeffd<04>Zddededee	fd<08>Z
dd	ed
edee	fd<0B>Zde
jdeeeffd
<0A>Zdedeefd<0F>Zde
jdeeeffd<10>Zdededefd<13>Z	ddeeefdeeefded
edee	f
d<17>Zdee	defd<1A>Zdeeeffd<1B>Zy)<1D>DiagramProcessorz8Processes diagrams from PDFs to extract meaning kernels.N<>configc<00>@<00>|xsi|_g|_ddddd<02>|_y)Nr)<04>pages_processed<65>diagrams_found<6E>kernels_extracted<65>errors)r7<00>kernels<6C>stats)rr7s  rrzDiagramProcessor.__init__Js)<00><00><1C>l<EFBFBD><02><04><0B>,.<2E><04><0C> <20><1F>!"<22><17>	
<EFBFBD><04>
r<00>pdf_path<74>
output_dirrc	<00><00>tstd<01><00>t|<01>}|j<00>st	d|<01><00><02><00>td|<01><00><02>|rt|<02>}n|j|j<00>d<04>z}|jdd<05><06>	ddl	m
}||d	<09>
<EFBFBD>}tdt|<05><00>d<0C><03>g}t|<05>D]y\}}	|dz}
td|
<EFBFBD>dt|<05><00><00><04>|d|
d<14>d<15>z}|	j|<0B>|j!||
<EFBFBD>}|j#|<0C>|jdxxdz
cc<<00>{|j%||<03>|S#t$r1}td
|<06><00><02>|jdxxdz
cc<gcYd}~Sd}~wwxYw)z(Extract meaning kernels from a PDF file.z(pdf2image is required for PDF processingzPDF not found: zProcessing PDF: <20>_kernelsT)<02>parents<74>exist_okr)<01>convert_from_pathi,)<01>dpiz
Converted z pages to imageszError converting PDF: r<<00>NzProcessing page <20>/<2F>page_<65>03d<33>.pngr9)<13>PDF2IMAGE_AVAILABLE<4C>ImportErrorr<00>exists<74>FileNotFoundError<6F>print<6E>parent<6E>stem<65>mkdir<69>	pdf2imagerE<00>len<65>	Exceptionr><00>	enumerate<74>save<76>extract_from_image<67>extend<6E>
_save_kernels)
rr?r@<00>output_pathrE<00>images<65>e<>all_kernels<6C>i<>image<67>page_num<75>temp_image_path<74>page_kernelss
             r<00>extract_from_pdfz!DiagramProcessor.extract_from_pdfTs<><00><00>"<22><1D>H<>I<>I<><17><08>><3E><08><17><EFBFBD><EFBFBD> <20>#<23>o<EFBFBD>h<EFBFBD>Z<EFBFBD>$@<40>A<>A<>
<0A> <20><18>
<EFBFBD>+<2B>,<2C><16><1E>z<EFBFBD>*<2A>K<EFBFBD>"<22>/<2F>/<2F>x<EFBFBD>}<7D>}<7D>o<EFBFBD>X<EFBFBD>,F<>F<>K<EFBFBD><13><19><19>$<24><14><19>6<>	<16>3<>&<26>x<EFBFBD>S<EFBFBD>9<>F<EFBFBD><11>J<EFBFBD>s<EFBFBD>6<EFBFBD>{<7B>m<EFBFBD>+;<3B><<3C>=<3D><19><0B>!<21>&<26>)<29>	/<2F>H<EFBFBD>A<EFBFBD>u<EFBFBD><18>1<EFBFBD>u<EFBFBD>H<EFBFBD><11>$<24>X<EFBFBD>J<EFBFBD>a<EFBFBD><03>F<EFBFBD><0B>}<7D>=<3D>><3E>*<2A>e<EFBFBD>H<EFBFBD>S<EFBFBD>><3E><14>,F<>F<>O<EFBFBD><11>J<EFBFBD>J<EFBFBD><EFBFBD>'<27> <20>2<>2<>?<3F>H<EFBFBD>M<>L<EFBFBD><17><1E><1E>|<7C>,<2C><10>J<EFBFBD>J<EFBFBD>(<28>)<29>Q<EFBFBD>.<2E>)<29>	/<2F>	
<0A><1A><1A>;<3B><0B>4<><1A><1A><>1<19>	<16><11>*<2A>1<EFBFBD>#<23>.<2E>/<2F><10>J<EFBFBD>J<EFBFBD>x<EFBFBD> <20>A<EFBFBD>%<25> <20><15>I<EFBFBD><49>	<16>s<00>(E<00>	F<03>&F<03>;F<03>F<03>
image_pathrbc<00>(<00>tstd<01><00>t|<01>}|j<00>st	d|<01><00><02><00>td|<01><00><02>	t
j|<01>}|j|<03>}|j|<03>}|j||t|<01>|<02>}|jdxxdz
cc<|jd	xxt|<07>z
cc<|S#t$r1}td|<04><00><02>|jdxxdz
cc<gcYd}~Sd}~wwxYw)
z&Extract meaning kernels from an image.z$PIL is required for image processingzImage not found: zProcessing image: zError loading image: r<rGNr:r;)<0F>
PIL_AVAILABLErMrrNrOrPr	<00>openrVr><00>_extract_text_from_image<67>_analyze_image_structure<72>_generate_kernelsr2rU)rrfrbrar^<00>extracted_text<78>structure_analysisr=s        rrYz#DiagramProcessor.extract_from_image<67>s<00><00><1C><1D>D<>E<>E<><19>*<2A>%<25>
<EFBFBD><19> <20> <20>"<22>#<23>&7<>
<EFBFBD>|<7C>$D<>E<>E<>
<0A>"<22>:<3A>,<2C>/<2F>0<>	<16><19>J<EFBFBD>J<EFBFBD>z<EFBFBD>*<2A>E<EFBFBD><1E>6<>6<>u<EFBFBD>=<3D><0E>"<22>:<3A>:<3A>5<EFBFBD>A<><1A><17>(<28>(<28><1A><1E><0F>
<EFBFBD>O<EFBFBD><14>	
<EFBFBD><07>	
<0A>
<EFBFBD>
<EFBFBD>#<23>$<24><01>)<29>$<24><0C>
<EFBFBD>
<EFBFBD>&<26>'<27>3<EFBFBD>w<EFBFBD><<3C>7<>'<27><16><0E><>-<19>	<16><11>)<29>!<21><13>-<2D>.<2E><10>J<EFBFBD>J<EFBFBD>x<EFBFBD> <20>A<EFBFBD>%<25> <20><15>I<EFBFBD><49>	<16>s<00>
C<00>	D<03> &D<03>D<03>Drac<00>$<00>dgdgd<03>}tr<>	tj|tjj<00><04>}g}g}t|d<00>D]K\}}t
|d|<00>dkDs<01>|j|<07>|jt
|d|<00><00><00>Mdj|<04>|d	<|j|<03>|d
<|rt|<05>t|<05>znd|d<||d<|S|S#t$r}td
|<08><00><02>Yd}~|Sd}~wwxYw)z"Extract text from image using OCR.<2E>r-)<04>	full_text<78>linesr<00>words)<01>output_type<70>text<78>confr<00> rqrrrrszOCR error: N)<0E>TESSERACT_AVAILABLE<4C>pytesseract<63>
image_to_data<74>Output<75>DICTrW<00>int<6E>append<6E>join<69>_group_text_into_lines<65>sumrUrVrP)	rra<00>	text_data<74>data<74>texts<74>confidencesr`rur^s	         rrjz)DiagramProcessor._extract_text_from_image<67>s-<00><00><1C><17><1D><17>	
<EFBFBD>	<09><1F>
)<29>"<22>0<>0<><15>K<EFBFBD>DV<44>DV<44>D[<5B>D[<5B>\<5C><04><1B><05> <20><0B>(<28><14>f<EFBFBD><1C>6<>A<01>G<EFBFBD>A<EFBFBD>t<EFBFBD><1A>4<EFBFBD><06><<3C><01>?<3F>+<2B>a<EFBFBD>/<2F><1D><0C><0C>T<EFBFBD>*<2A>#<23>*<2A>*<2A>3<EFBFBD>t<EFBFBD>F<EFBFBD>|<7C>A<EFBFBD><EFBFBD>+?<3F>@<40>A<01>
*-<2D><18><18>%<25><1F>	<09>+<2B>&<26>%)<29>%@<40>%@<40><14>%F<>	<09>'<27>"<22>Q\<5C>#<23>k<EFBFBD>*:<3A>S<EFBFBD><1B>=M<>*M<>bc<62>	<09>,<2C>'<27>%*<2A>	<09>'<27>"<22>
<19><18>y<EFBFBD><18><><1D>
)<29><15><0B>A<EFBFBD>3<EFBFBD>'<27>(<28>(<28><18><18><>
)<29>s<00>AC-<00>+A>C-<00>-	D<03>6D
<03>
D<03>ocr_datac<00><><00>g}g}d}d}d}tt|d<00><00>D]}t|d|<00>dkr<01>|d|}|d|}	|d|}
||k7s
|	|k7s|
|k7r*|r |jdj	|<03><00>g}|}|	}|
}|j|d|<00><00><>|r |jdj	|<03><00>|S)	zGroup OCR words into lines.<2E><><EFBFBD><EFBFBD><EFBFBD>rurvr<00>	block_num<75>par_num<75>line_numrw)<05>rangerUr}r~r)rr<>rr<00>current_line<6E>
current_block<63>current_par<61>current_line_numr`r<>r<>r<>s           rr<>z'DiagramProcessor._group_text_into_lines<65>s<><00><00><12><05><19><0C><1A>
<0A><18><0B><1D><18><16>s<EFBFBD>8<EFBFBD>F<EFBFBD>+<2B>,<2C>-<2D>	5<>A<EFBFBD><12>8<EFBFBD>F<EFBFBD>#<23>A<EFBFBD>&<26>'<27>1<EFBFBD>,<2C><18> <20><1B>-<2D>a<EFBFBD>0<>I<EFBFBD><1E>y<EFBFBD>)<29>!<21>,<2C>G<EFBFBD><1F>
<EFBFBD>+<2B>A<EFBFBD>.<2E>H<EFBFBD><19>]<5D>*<2A><17>;<3B>&<26><18>,<2C>,<2C><1F><19>L<EFBFBD>L<EFBFBD><13><18><18>,<2C>!7<>8<>!<21><0C> )<29>
<0A>%<25><0B>#+<2B> <20><18><1F><1F><08><16> 0<><11> 3<>4<>'	5<>*<18><11>L<EFBFBD>L<EFBFBD><13><18><18>,<2C>/<2F>0<><14>rc<00>r<00>|j\}}||z}||d<01>||dkD|dk|j||<03>dd<04>}|S)z-Analyze image structure (simplified version).)<02>width<74>heightrG<00>medium)<06>
dimensions<EFBFBD>aspect_ratio<69>is_landscape<70>is_portrait<69>estimated_diagram_type<70>
complexity)<02>size<7A>_estimate_diagram_type)rrar<>r<>r<><00>analysiss      rrkz)DiagramProcessor._analyze_image_structure<72>sX<00><00>
<1E>
<EFBFBD>
<EFBFBD>
<0A><05>v<EFBFBD><1C>v<EFBFBD>~<7E><0C>%*<2A>V<EFBFBD><<3C>(<28>(<28>1<EFBFBD>,<2C>'<27>!<21>+<2B>&*<2A>&A<>&A<>%<25><16>&P<>"<22>
<EFBFBD><08><18>rr<>r<>c<00>B<00>||z}|dkDry|dkryd|cxkrdkryyy)	z7Estimate diagram type based on dimensions (simplified).<2E><00>	flowchartg<00>?<3F>vertical_hierarchy皙<79><E79A99><EFBFBD><EFBFBD><EFBFBD>?g333333<33>?<3F>square_diagram<61>standard_diagramr4)rr<>r<>r<>s    rr<>z'DiagramProcessor._estimate_diagram_types;<00><00><1C>v<EFBFBD>~<7E><0C><17>!<21><1B><1E>
<19>C<EFBFBD>
<1F>'<27>
<10>L<EFBFBD>
'<27>C<EFBFBD>
'<27>#<23>(<28>&rr<><00>	structurerc<00><><00>g}dtj<00>jd<02><00><00>}|r|d|<04><00>z
}|dj<00>rJt	|<06>d<05>|d||ddzdt|d	<00>t|d
<00>|d<0B><04><0C>}|j
|<07>d
|d<00>d<0F>}|d|dd<00>d|dd<00>d<0F>z
}|d|dd<17>d<0F>z
}|d|drdn|drdnd<1D>d<1E>z
}t	|<06>d<1F>||d d!|d"<22><02><0C>}	|j
|	<09>|dj<00>rOd#|d<00>d$<24>}
|
d%|dd&d'<00>d(<28>z
}
t	|<06>d)<29>|
|d*d+t|d<00>|dd,<2C><03><0C>}|j
|<0B>|jj|<05>|S)-z-Generate meaning kernels from extracted data.<2E>kernel_z
%Y%m%d_%H%M%S<>_prq<00>_textrgY@<40>text_extractionrsrr)<04>type<70>
word_count<EFBFBD>
line_countr<EFBFBD>)rr
rrrzDiagram type: r<>z. zDimensions: r<>r<><00>xr<78>zAspect ratio: r<><00>.2fz
Orientation: r<><00>	landscaper<65><00>portrait<69>square<72>.<2E>
_structurer<EFBFBD>rn)r<>r<>zResearch diagram analysis: z with text content. zKey elements: N<><4E>r+<00>_summarygffffff<66>?<3F>summary)r<><00>text_length<74>structure_type)	rr<00>strftime<6D>striprrUr~r=rZ)rr<>r<>rrbr=<00>base_id<69>text_kernel<65>structure_content<6E>structure_kernelr<6C><00>summary_kernels            rrlz"DiagramProcessor._generate_kernelss><00><00>
<15><07><1C>H<EFBFBD>L<EFBFBD>L<EFBFBD>N<EFBFBD>3<>3<>O<EFBFBD>D<>E<>F<><07><13><13><12>H<EFBFBD>:<3A><EFBFBD>&<26>G<EFBFBD><15>[<5B>!<21>'<27>'<27>)<29>'<27>$<24>I<EFBFBD>U<EFBFBD>+<2B>!<21>+<2B>.<2E><1D>$<24>\<5C>2<>U<EFBFBD>:<3A>-<2D>"%<25>i<EFBFBD><07>&8<>"9<>"%<25>i<EFBFBD><07>&8<>"9<>!*<2A>	<12><0E>K<EFBFBD>
<14>N<EFBFBD>N<EFBFBD>;<3B>'<27>-<2D>Y<EFBFBD>7O<37>-P<>,Q<>QS<51>T<><19><19>|<7C>I<EFBFBD>l<EFBFBD>,C<>G<EFBFBD>,L<>+M<>Q<EFBFBD>y<EFBFBD>Ye<59>Of<4F>go<67>Op<4F>Nq<4E>qs<71>t<>t<><19><19>~<7E>i<EFBFBD><0E>.G<><03>-L<>B<EFBFBD>O<>O<><19><19>}<7D>I<EFBFBD>n<EFBFBD><U<>[<5B>ir<69>tA<02>jB<02>[e<>HP<02>-Q<02>QR<02>S<02>	S<02><19>(<28> <20>	<09><1A>,<2C>%<25><19><1A>,<2C>%<25><0E>	
<EFBFBD><18>	<10><0E><0E>'<27>(<28><15>[<5B>!<21>'<27>'<27>)<29>3<>I<EFBFBD>>V<>4W<34>3X<33>Xl<58>m<>G<EFBFBD><13><1E>	<09>+<2B>(><3E>t<EFBFBD><03>(D<>'E<>S<EFBFBD>I<>I<>G<EFBFBD>*<2A>$<24>I<EFBFBD>X<EFBFBD>.<2E><1F><1D><1E>%<25>#&<26>y<EFBFBD><1B>'=<3D>#><3E>&/<2F>0H<30>&I<><12>
<0E>N<EFBFBD>
<14>N<EFBFBD>N<EFBFBD>><3E>*<2A>	
<0A><0C><0C><1B><1B>G<EFBFBD>$<24><16>rr=r\c	<00><><00>|std<01>y|dz}|D<00>cgc]}|j<00><00><02>}}t|d<04>5}tj||d<05><06>ddd<02>|dz}t|d<04>5}|jd<08>|jd	t
j<00>j<00><00>d
<EFBFBD><03>|jdt|<01><00>d<0C><03>|D]<5D>}|jd
|j<00>d
<EFBFBD><03>|jd|j<00>d
<EFBFBD><03>|jd|jd<10>d
<EFBFBD><03>|jd|j<00>d
<EFBFBD><03>|jd|j<00>d
<EFBFBD><03>|jd|j<00>d
<EFBFBD><03>|jdtj |j"d<05><06><00>d<0C><03><00><>	ddd<02>|dz}	t|	d<04>5}tj|j$|d<05><06>ddd<02>tdt|<01><00>d|<02><00><04>td|<03><00><02>td|<07><00><02>td|	<09><00><02>ycc}w#1swY<00><02>xYw#1swY<00><>xYw#1swY<00>lxYw)zSave kernels to files.zNo kernels to saveNzmeaning_kernels.json<6F>wr<77>)<01>indentzmeaning_kernels.mdz$# Meaning Kernels Extraction Report
zGenerated: <20>
zTotal kernels: z

z## Kernel: z- **Source**: z- **Confidence**: r<>z- **Timestamp**: z- **Hash**: z- **Content**: z- **Metadata**: zextraction_stats.jsonzSaved z kernels to z
  - JSON: z  - Markdown: z  - Statistics: )rPr(ri<00>json<6F>dump<6D>writerrrrUrrrrrr
<00>dumpsrr>)
rr=r\<00>	json_path<74>k<>kernels_data<74>f<>md_path<74>kernel<65>
stats_paths
          rr[zDiagramProcessor._save_kernelsTsQ<00><00><16><11>&<26>'<27><12> <20>"8<>8<>	<09>-4<>5<><01><01>	<09>	<09><0B>5<><0C>5<>
<11>)<29>S<EFBFBD>
!<21>	1<>Q<EFBFBD><10>I<EFBFBD>I<EFBFBD>l<EFBFBD>A<EFBFBD>a<EFBFBD>0<>	1<><1E> 4<>4<><07>
<11>'<27>3<EFBFBD>
<1F>	X<01>1<EFBFBD>
<0A>G<EFBFBD>G<EFBFBD>;<3B>=<3D>
<0A>G<EFBFBD>G<EFBFBD>k<EFBFBD>(<28>,<2C>,<2C>.<2E>":<3A>":<3A>"<<3C>!=<3D>R<EFBFBD>@<40>A<>
<0A>G<EFBFBD>G<EFBFBD>o<EFBFBD>c<EFBFBD>'<27>l<EFBFBD>^<5E>4<EFBFBD>8<>9<>!<21>
X<01><06><11><07><07>+<2B>f<EFBFBD>&6<>&6<>%7<>r<EFBFBD>:<3A>;<3B><11><07><07>.<2E><16><1D><1D><0F>r<EFBFBD>:<3A>;<3B><11><07><07>,<2C>V<EFBFBD>-><3E>-><3E>s<EFBFBD>,C<>2<EFBFBD>F<>G<><11><07><07>+<2B>F<EFBFBD>,<<3C>,<<3C>+=<3D>R<EFBFBD>@<40>A<><11><07><07>,<2C>v<EFBFBD>{<7B>{<7B>m<EFBFBD>2<EFBFBD>6<>7<><11><07><07>/<2F>&<26>.<2E>.<2E>)9<><12><<3C>=<3D><11><07><07>*<2A>4<EFBFBD>:<3A>:<3A>f<EFBFBD>o<EFBFBD>o<EFBFBD>a<EFBFBD>+P<>*Q<>QU<51>V<>W<>
X<01>	X<01>!<21>#:<3A>:<3A>
<EFBFBD>
<11>*<2A>c<EFBFBD>
"<22>	/<2F>a<EFBFBD><10>I<EFBFBD>I<EFBFBD>d<EFBFBD>j<EFBFBD>j<EFBFBD>!<21>A<EFBFBD>.<2E>	/<2F>	<0E><06>s<EFBFBD>7<EFBFBD>|<7C>n<EFBFBD>L<EFBFBD><1B>
<0A>><3E>?<3F>
<0A>
<EFBFBD>9<EFBFBD>+<2B>&<26>'<27>
<0A><0E>w<EFBFBD>i<EFBFBD>(<28>)<29>
<0A> <20><1A><0C>-<2D>.<2E><>;6<>	1<>	1<><31>
	X<01>	X<01><> 	/<2F>	/<2F>s)<00>I<04>I<03>/EI&<03>$#I2<03>I#<07>&I/<07>2I;c<00>6<00>|jj<00>S)zGet processing statistics.)r><00>copyr's r<00>	get_statszDiagramProcessor.get_stats{s<00><00><13>z<EFBFBD>z<EFBFBD><EFBFBD><EFBFBD> <20> rr)r.r/r0r1rr2rrrrrer}rYr	rjr<>rkr<>rlrr[r<>r4rrr6r6GsI<00><00>B<>
<EFBFBD>t<EFBFBD>C<EFBFBD><13>H<EFBFBD>~<7E>
<EFBFBD>/<1B><13>/<1B>#<23>/<1B><14>m<EFBFBD>I\<5C>/<1B>b$<17>S<EFBFBD>$<17>C<EFBFBD>$<17>4<EFBFBD>P]<5D>K^<5E>$<17>L<19>e<EFBFBD>k<EFBFBD>k<EFBFBD><19>d<EFBFBD>3<EFBFBD><03>8<EFBFBD>n<EFBFBD><19>B <15>t<EFBFBD> <15><04>S<EFBFBD>	<09> <15>D<18>e<EFBFBD>k<EFBFBD>k<EFBFBD><18>d<EFBFBD>3<EFBFBD><03>8<EFBFBD>n<EFBFBD><18>(&<26>C<EFBFBD>&<26><13>&<26><13>&<26> *.<2E>C<17>4<EFBFBD><03>S<EFBFBD><08>><3E>C<17>$(<28><13>c<EFBFBD><18>N<EFBFBD>C<17>!$<24>C<17>$'<27>C<17>37<33>}<7D>2E<32>C<17>J%/<2F>T<EFBFBD>-<2D>%8<>%/<2F>t<EFBFBD>%/<2F>N!<21>4<EFBFBD><03>S<EFBFBD><08>><3E>!rr6c<00><00>tjd<01><02>}|jdd<04><05>|jddd<08><05>|jd	d
d<0B><05>|jdd
dd<0F><10>|j<00>}i}|jr3t|j<00>5}t
j|<03>}ddd<11>t|<02>}t|j<00>}|j<00>r<>|jj<00>dk(r|j||j<00>}<06>n2|jj<00>dvr|j!|<05>}<06>nt#d|j<00><00><02>t%j&d<15>n<>|j)<00>r<>g}|j+<00>D]<5D>}|jj<00>dk(r.|j||j<00>}|j-|<06><00>N|jj<00>dvs<01>k|j!|<08>}|j-|<06><00><>n#t#d|<05><00><02>t%j&d<15>|j/<00>}	t#d<17>t#d<18>t#d<19>t#d|	d<00><00><02>t#d|	d<00><00><02>t#d|	d<00><00><02>t#d |	d!<00><00><02>t#d<19>t%j&|	d!d"k(rd"<22>yd<15>y#1swY<00><02>LxYw)#z(Command line interface for the pipeline.z.Extract meaning kernels from research diagrams)<01>description<6F>inputz!Input PDF or image file/directory)<01>helpz-oz--outputzOutput directoryz-cz--configzConfiguration file (JSON)z-vz	--verbose<73>
store_truezVerbose output)<02>actionr<6E>Nz.pdf)rKz.jpgz.jpegz.tiffz.bmpzUnsupported file type: rGzInput not found: z3
==================================================zEXTRACTION SUMMARYz2==================================================zPages processed: r9zDiagrams found: r:zKernels extracted: r;zErrors: r<r)<18>argparse<73>ArgumentParser<65>add_argument<6E>
parse_argsr7rir<><00>loadr6rr<><00>is_file<6C>suffix<69>lowerre<00>outputrYrP<00>sys<79>exit<69>is_dir<69>iterdirrZr<>)
<EFBFBD>parser<65>argsr7r<><00>	processor<6F>
input_pathr=r_<00>	file_pathr>s
          r<00>mainr<6E>s<><00><00>
<15>
$<24>
$<24>1a<31>
b<>F<EFBFBD>
<EFBFBD><17><17><07>&I<><17>J<>
<EFBFBD><17><17><04>j<EFBFBD>/A<><17>B<>
<EFBFBD><17><17><04>j<EFBFBD>/J<><17>K<>
<EFBFBD><17><17><04>k<EFBFBD>,<2C>EU<45><17>V<><11><1C><1C><1E>D<EFBFBD><10>F<EFBFBD><0B>{<7B>{<7B>
<11>$<24>+<2B>+<2B>
<1E>	"<22>!<21><19>Y<EFBFBD>Y<EFBFBD>q<EFBFBD>\<5C>F<EFBFBD>	"<22>!<21><16>(<28>I<EFBFBD><16>d<EFBFBD>j<EFBFBD>j<EFBFBD>!<21>J<EFBFBD><11><19><19><1B><15><1C><1C>"<22>"<22>$<24><06>.<2E><1F>0<>0<><1A>T<EFBFBD>[<5B>[<5B>I<>G<EFBFBD>
<17>
<1E>
<1E>
$<24>
$<24>
&<26>*T<>
T<><1F>2<>2<>:<3A>><3E>G<EFBFBD><11>+<2B>J<EFBFBD>,=<3D>,=<3D>+><3E>?<3F>@<40><0F>H<EFBFBD>H<EFBFBD>Q<EFBFBD>K<EFBFBD>	<13>	<1A>	<1A>	<1C><18><0B>#<23>+<2B>+<2B>-<2D>	,<2C>I<EFBFBD><18><1F><1F>%<25>%<25>'<27>6<EFBFBD>1<>#<23>4<>4<>Y<EFBFBD><04><0B><0B>L<><07><1B>"<22>"<22>7<EFBFBD>+<2B><1A>!<21>!<21>'<27>'<27>)<29>-W<>W<>#<23>6<>6<>y<EFBFBD>A<><07><1B>"<22>"<22>7<EFBFBD>+<2B>
	,<2C>	<0E>!<21>*<2A><1C>.<2E>/<2F><0B><08><08><11><0B>
<16><1F><1F>!<21>E<EFBFBD>	<09>-<2D><18>	<09>
<1E><1F>	<09>&<26>M<EFBFBD>	<09><1D>e<EFBFBD>$5<>6<>7<>
8<EFBFBD>9<>	<09><1C>U<EFBFBD>#3<>4<>5<>
6<EFBFBD>7<>	<09><1F><05>&9<> :<3A>;<3B>
<<3C>=<3D>	<09>H<EFBFBD>U<EFBFBD>8<EFBFBD>_<EFBFBD>%<25>
&<26>'<27>	<09>&<26>M<EFBFBD><08>H<EFBFBD>H<EFBFBD>%<25><08>/<2F>Q<EFBFBD>&<26>Q<EFBFBD>.<2E>A<EFBFBD>.<2E>W	"<22>	"<22>s<00>K:<03>:L<07>__main__)r1<00>osr<73>r<>r<><00>pathlibrr<00>typingrrrrr <00>PILr	rhrMrPryrxrTrLrr6r<>r.r4rr<00><module>r<>s<><00><01><04>

<EFBFBD>
<EFBFBD><0B><0F><18><1D>,<2C>,<2C><0E>J<01><19><18>M<EFBFBD>
W<01><16><1E><17>
S<01><14><1E><17>
D<01>D<01>Bv!<21>v!<21>p	8/<2F>t<0C>z<EFBFBD><19><08>F<EFBFBD><1A><>K
<13>J<01><19>M<EFBFBD>	<09>
H<EFBFBD>I<>J<01><><13>W<01><1F><17>	<09>
U<EFBFBD>V<>W<01><><13>S<01><1F><17>	<09>
Q<EFBFBD>R<>S<01>s4<00>A+<00>B<00>B<00>+A=<03><A=<03>B<03>B<03>B'<03>&B'
-												Fix #493: Add multimodal meaning kernel extraction pipeline

- Added extract_meaning_kernels.py for processing PDF diagrams
- Extracts text using OCR (Tesseract) when available
- Analyzes diagram structure (type, dimensions, orientation)
- Generates structured meaning kernels with metadata
- Outputs JSON (machine-readable) and Markdown (human-readable)
- Includes test pipeline and documentation
- Supports single files and batch processing

Pipeline components:
- DiagramProcessor: Main processing engine
- MeaningKernel: Structured kernel representation
- PDF to image conversion
- OCR text extraction
- Structure analysis
- Kernel generation with confidence scoring

Acceptance criteria met:
✓ Processes academic PDF diagrams
✓ Extracts structured text meaning kernels
✓ Generates machine-readable JSON output
✓ Includes human-readable reports
✓ Supports batch processing
✓ Provides confidence scoring

											
										
										
											2026-04-13 21:20:42 -04:00
+								<EFBFBD>
 								s<><73>iR?<00><00>T<00>dZddlZddlZddlZddlZddlmZddlmZddlm	Z	m
 								Z
 								mZmZddl
Z
	ddlmZdZ	ddlZdZ	ddlZdZGd<0C>d
<0A>ZGd<0E>d<0F>Zd<10>Zedk(re<19>yy#e$r
dZed	<09>Y<00>DwxYw#e$r
dZed
 								<EFBFBD>Y<00>RwxYw#e$r
dZed<0B>Y<00>`wxYw)z<>
 								Multimodal Meaning Kernel Extraction Pipeline
 								Extracts structured meaning kernels from academic PDF diagrams.
 								Issue #493: [Multimodal] Extract Meaning Kernels from Research Diagrams
 								<EFBFBD>N)<01>Path)<01>datetime)<04>List<73>Dict<63>Any<6E>Optional)<01>ImageTFz<Warning: PIL not available. Install with: pip install PillowzIWarning: pytesseract not available. Install with: pip install pytesseractzEWarning: pdf2image not available. Install with: pip install pdf2imagec
<00>h<00>eZdZdZ	d
dededededeeeff
 								d<08>Zd	efd
 								<EFBFBD>Z	d	eeeffd<0B>Z
 								d	efd<0C>Zy)<0E>
MeaningKernelz6Represents an extracted meaning kernel from a diagram.N<>	kernel_id<69>content<6E>source<63>
 								confidence<EFBFBD>metadatac<00><><00>||_||_||_||_|xsi|_tj<00>j<00>|_|j<00>|_
 								y<00>N)rr
rrrr<00>now<6F>	isoformat<61>	timestamp<6D>_generate_hash<73>hash)<06>selfrr
rrrs      <20>L/private/tmp/timmy-config-burn/scripts/multimodal/extract_meaning_kernels.py<70>__init__zMeaningKernel.__init__)sO<00><00>"<22><04><0E><1E><04><0C><1C><04><0B>$<24><04><0F> <20><0E>B<EFBFBD><04>
<0A>!<21><1C><1C><1E>1<>1<>3<><04><0E><18>'<27>'<27>)<29><04>	<09><00>returnc<00><><00>|j<00>d|j<00>d|j<00>d|j<00><00>}t	j
 								|j
<00><00>j<00>ddS)z'Generate a unique hash for this kernel.<2E>:N<>)rr
rr<00>hashlib<69>sha256<35>encode<64>	hexdigest)r<00>content_strs  rrzMeaningKernel._generate_hash3sX<00><00><1D><1E><1E>(<28><01>$<24>,<2C>,<2C><1E>q<EFBFBD><14><1B><1B>
<0A>Q<EFBFBD>t<EFBFBD>~<7E>~<7E>FV<46>W<><0B><16>~<7E>~<7E>k<EFBFBD>0<>0<>2<>3<>=<3D>=<3D>?<3F><03><12>D<>Drc<00><><00>|j|j|j|j|j|j
 								|jd<01>S)z(Convert to dictionary for serialization.<2E>rr
rrrrrr&<00>rs r<00>to_dictzMeaningKernel.to_dict8s?<00><00><1E><1E><1E><1B>|<7C>|<7C><1A>k<EFBFBD>k<EFBFBD><1E>/<2F>/<2F><1C>
<0A>
<0A><1D><1E><1E><18>I<EFBFBD>I<EFBFBD>
 								<EFBFBD>
 								rc<00>B<00>d|j<00>d|jdd<00>d<04>S)NzKernel[z]: <20>d<00>...)rr
r's r<00>__str__zMeaningKernel.__str__Ds(<00><00><18><14><1E><1E>(<28><03>D<EFBFBD>L<EFBFBD>L<EFBFBD><14>#<23>,><3E>+?<3F>s<EFBFBD>C<>Cr)<02>N)<0C>__name__<5F>
 								__module__<EFBFBD>__qualname__<5F>__doc__<5F>str<74>floatrrrrr(r,<00>rrrr&sn<00><00>@<40>FJ<01>*<2A>#<23>*<2A><03>*<2A>S<EFBFBD>*<2A>"<22>*<2A>48<34><13>c<EFBFBD><18>N<EFBFBD>*<2A>E<01><03>E<01>
 								<EFBFBD><14>c<EFBFBD>3<EFBFBD>h<EFBFBD><1E>
 								<EFBFBD>D<01><13>Drrc<00>L<00>eZdZdZddeeeffd<04>Zddededee	fd<08>Z
 								dd	ed
 								edee	fd<0B>Zde
jdeeeffd
<0A>Zdedeefd<0F>Zde
jdeeeffd<10>Zdededefd<13>Z	ddeeefdeeefded
 								edee	f
 								d<17>Zdee	defd<1A>Zdeeeffd<1B>Zy)<1D>DiagramProcessorz8Processes diagrams from PDFs to extract meaning kernels.N<>configc<00>@<00>|xsi|_g|_ddddd<02>|_y)Nr)<04>pages_processed<65>diagrams_found<6E>kernels_extracted<65>errors)r7<00>kernels<6C>stats)rr7s  rrzDiagramProcessor.__init__Js)<00><00><1C>l<EFBFBD><02><04><0B>,.<2E><04><0C> <20><1F>!"<22><17>
 								<EFBFBD><04>
 								r<00>pdf_path<74>
 								output_dirrc	<00><00>tstd<01><00>t|<01>}|j<00>st	d|<01><00><02><00>td|<01><00><02>|rt|<02>}n|j|j<00>d<04>z}|jdd<05><06>	ddl	m
 								}||d	<09>
 								<EFBFBD>}tdt|<05><00>d<0C><03>g}t|<05>D]y\}}	|dz}
 								td|
 								<EFBFBD>dt|<05><00><00><04>|d|
 								d<14>d<15>z}|	j|<0B>|j!||
 								<EFBFBD>}|j#|<0C>|jdxxdz
cc<<00>{|j%||<03>|S#t$r1}td
|<06><00><02>|jdxxdz
cc<gcYd}~Sd}~wwxYw)z(Extract meaning kernels from a PDF file.z(pdf2image is required for PDF processingzPDF not found: zProcessing PDF: <20>_kernelsT)<02>parents<74>exist_okr)<01>convert_from_pathi,)<01>dpiz
 								Converted z pages to imageszError converting PDF: r<<00>NzProcessing page <20>/<2F>page_<65>03d<33>.pngr9)<13>PDF2IMAGE_AVAILABLE<4C>ImportErrorr<00>exists<74>FileNotFoundError<6F>print<6E>parent<6E>stem<65>mkdir<69>	pdf2imagerE<00>len<65>	Exceptionr><00>	enumerate<74>save<76>extract_from_image<67>extend<6E>
_save_kernels)
rr?r@<00>output_pathrE<00>images<65>e<>all_kernels<6C>i<>image<67>page_num<75>temp_image_path<74>page_kernelss
             r<00>extract_from_pdfz!DiagramProcessor.extract_from_pdfTs<><00><00>"<22><1D>H<>I<>I<><17><08>><3E><08><17><EFBFBD><EFBFBD> <20>#<23>o<EFBFBD>h<EFBFBD>Z<EFBFBD>$@<40>A<>A<>
<0A> <20><18>
 								<EFBFBD>+<2B>,<2C><16><1E>z<EFBFBD>*<2A>K<EFBFBD>"<22>/<2F>/<2F>x<EFBFBD>}<7D>}<7D>o<EFBFBD>X<EFBFBD>,F<>F<>K<EFBFBD><13><19><19>$<24><14><19>6<>	<16>3<>&<26>x<EFBFBD>S<EFBFBD>9<>F<EFBFBD><11>J<EFBFBD>s<EFBFBD>6<EFBFBD>{<7B>m<EFBFBD>+;<3B><<3C>=<3D><19><0B>!<21>&<26>)<29>	/<2F>H<EFBFBD>A<EFBFBD>u<EFBFBD><18>1<EFBFBD>u<EFBFBD>H<EFBFBD><11>$<24>X<EFBFBD>J<EFBFBD>a<EFBFBD><03>F<EFBFBD><0B>}<7D>=<3D>><3E>*<2A>e<EFBFBD>H<EFBFBD>S<EFBFBD>><3E><14>,F<>F<>O<EFBFBD><11>J<EFBFBD>J<EFBFBD><EFBFBD>'<27> <20>2<>2<>?<3F>H<EFBFBD>M<>L<EFBFBD><17><1E><1E>|<7C>,<2C><10>J<EFBFBD>J<EFBFBD>(<28>)<29>Q<EFBFBD>.<2E>)<29>	/<2F>	
<0A><1A><1A>;<3B><0B>4<><1A><1A><>1<19>	<16><11>*<2A>1<EFBFBD>#<23>.<2E>/<2F><10>J<EFBFBD>J<EFBFBD>x<EFBFBD> <20>A<EFBFBD>%<25> <20><15>I<EFBFBD><49>	<16>s<00>(E<00>	F<03>&F<03>;F<03>F<03>
 								image_pathrbc<00>(<00>tstd<01><00>t|<01>}|j<00>st	d|<01><00><02><00>td|<01><00><02>	t
j|<01>}|j|<03>}|j|<03>}|j||t|<01>|<02>}|jdxxdz
cc<|jd	xxt|<07>z
cc<|S#t$r1}td|<04><00><02>|jdxxdz
cc<gcYd}~Sd}~wwxYw)
 								z&Extract meaning kernels from an image.z$PIL is required for image processingzImage not found: zProcessing image: zError loading image: r<rGNr:r;)<0F>
PIL_AVAILABLErMrrNrOrPr	<00>openrVr><00>_extract_text_from_image<67>_analyze_image_structure<72>_generate_kernelsr2rU)rrfrbrar^<00>extracted_text<78>structure_analysisr=s        rrYz#DiagramProcessor.extract_from_image<67>s<00><00><1C><1D>D<>E<>E<><19>*<2A>%<25>
 								<EFBFBD><19> <20> <20>"<22>#<23>&7<>
 								<EFBFBD>|<7C>$D<>E<>E<>
<0A>"<22>:<3A>,<2C>/<2F>0<>	<16><19>J<EFBFBD>J<EFBFBD>z<EFBFBD>*<2A>E<EFBFBD><1E>6<>6<>u<EFBFBD>=<3D><0E>"<22>:<3A>:<3A>5<EFBFBD>A<><1A><17>(<28>(<28><1A><1E><0F>
 								<EFBFBD>O<EFBFBD><14>
 								<EFBFBD><07>	
<0A>
 								<EFBFBD>
 								<EFBFBD>#<23>$<24><01>)<29>$<24><0C>
 								<EFBFBD>
 								<EFBFBD>&<26>'<27>3<EFBFBD>w<EFBFBD><<3C>7<>'<27><16><0E><>-<19>	<16><11>)<29>!<21><13>-<2D>.<2E><10>J<EFBFBD>J<EFBFBD>x<EFBFBD> <20>A<EFBFBD>%<25> <20><15>I<EFBFBD><49>	<16>s<00>
 								C<00>	D<03> &D<03>D<03>Drac<00>$<00>dgdgd<03>}tr<>	tj|tjj<00><04>}g}g}t|d<00>D]K\}}t
|d|<00>dkDs<01>|j|<07>|jt
|d|<00><00><00>Mdj|<04>|d	<|j|<03>|d
 								<|rt|<05>t|<05>znd|d<||d<|S|S#t$r}td
|<08><00><02>Yd}~|Sd}~wwxYw)z"Extract text from image using OCR.<2E>r-)<04>	full_text<78>linesr<00>words)<01>output_type<70>text<78>confr<00> rqrrrrszOCR error: N)<0E>TESSERACT_AVAILABLE<4C>pytesseract<63>
image_to_data<74>Output<75>DICTrW<00>int<6E>append<6E>join<69>_group_text_into_lines<65>sumrUrVrP)	rra<00>	text_data<74>data<74>texts<74>confidencesr`rur^s	         rrjz)DiagramProcessor._extract_text_from_image<67>s-<00><00><1C><17><1D><17>
 								<EFBFBD>	<09><1F>
)<29>"<22>0<>0<><15>K<EFBFBD>DV<44>DV<44>D[<5B>D[<5B>\<5C><04><1B><05> <20><0B>(<28><14>f<EFBFBD><1C>6<>A<01>G<EFBFBD>A<EFBFBD>t<EFBFBD><1A>4<EFBFBD><06><<3C><01>?<3F>+<2B>a<EFBFBD>/<2F><1D><0C><0C>T<EFBFBD>*<2A>#<23>*<2A>*<2A>3<EFBFBD>t<EFBFBD>F<EFBFBD>|<7C>A<EFBFBD><EFBFBD>+?<3F>@<40>A<01>
 								*-<2D><18><18>%<25><1F>	<09>+<2B>&<26>%)<29>%@<40>%@<40><14>%F<>	<09>'<27>"<22>Q\<5C>#<23>k<EFBFBD>*:<3A>S<EFBFBD><1B>=M<>*M<>bc<62>	<09>,<2C>'<27>%*<2A>	<09>'<27>"<22>
 								<19><18>y<EFBFBD><18><><1D>
)<29><15><0B>A<EFBFBD>3<EFBFBD>'<27>(<28>(<28><18><18><>
)<29>s<00>AC-<00>+A>C-<00>-	D<03>6D
 								<03>
 								D<03>ocr_datac<00><><00>g}g}d}d}d}tt|d<00><00>D]}t|d|<00>dkr<01>|d|}|d|}	|d|}
 								||k7s
 								|	|k7s|
 								|k7r*|r |jdj	|<03><00>g}|}|	}|
 								}|j|d|<00><00><>|r |jdj	|<03><00>|S)	zGroup OCR words into lines.<2E><><EFBFBD><EFBFBD><EFBFBD>rurvr<00>	block_num<75>par_num<75>line_numrw)<05>rangerUr}r~r)rr<>rr<00>current_line<6E>
current_block<63>current_par<61>current_line_numr`r<>r<>r<>s           rr<>z'DiagramProcessor._group_text_into_lines<65>s<><00><00><12><05><19><0C><1A>
<0A><18><0B><1D><18><16>s<EFBFBD>8<EFBFBD>F<EFBFBD>+<2B>,<2C>-<2D>	5<>A<EFBFBD><12>8<EFBFBD>F<EFBFBD>#<23>A<EFBFBD>&<26>'<27>1<EFBFBD>,<2C><18> <20><1B>-<2D>a<EFBFBD>0<>I<EFBFBD><1E>y<EFBFBD>)<29>!<21>,<2C>G<EFBFBD><1F>
 								<EFBFBD>+<2B>A<EFBFBD>.<2E>H<EFBFBD><19>]<5D>*<2A><17>;<3B>&<26><18>,<2C>,<2C><1F><19>L<EFBFBD>L<EFBFBD><13><18><18>,<2C>!7<>8<>!<21><0C> )<29>
<0A>%<25><0B>#+<2B> <20><18><1F><1F><08><16> 0<><11> 3<>4<>'	5<>*<18><11>L<EFBFBD>L<EFBFBD><13><18><18>,<2C>/<2F>0<><14>rc<00>r<00>|j\}}||z}||d<01>||dkD|dk|j||<03>dd<04>}|S)z-Analyze image structure (simplified version).)<02>width<74>heightrG<00>medium)<06>
 								dimensions<EFBFBD>aspect_ratio<69>is_landscape<70>is_portrait<69>estimated_diagram_type<70>
 								complexity)<02>size<7A>_estimate_diagram_type)rrar<>r<>r<><00>analysiss      rrkz)DiagramProcessor._analyze_image_structure<72>sX<00><00>
 								<1E>
 								<EFBFBD>
 								<EFBFBD>
<0A><05>v<EFBFBD><1C>v<EFBFBD>~<7E><0C>%*<2A>V<EFBFBD><<3C>(<28>(<28>1<EFBFBD>,<2C>'<27>!<21>+<2B>&*<2A>&A<>&A<>%<25><16>&P<>"<22>
 								<EFBFBD><08><18>rr<>r<>c<00>B<00>||z}|dkDry|dkryd|cxkrdkryyy)	z7Estimate diagram type based on dimensions (simplified).<2E><00>	flowchartg<00>?<3F>vertical_hierarchy皙<79><E79A99><EFBFBD><EFBFBD><EFBFBD>?g333333<33>?<3F>square_diagram<61>standard_diagramr4)rr<>r<>r<>s    rr<>z'DiagramProcessor._estimate_diagram_types;<00><00><1C>v<EFBFBD>~<7E><0C><17>!<21><1B><1E>
<19>C<EFBFBD>
<1F>'<27>
<10>L<EFBFBD>
'<27>C<EFBFBD>
'<27>#<23>(<28>&rr<><00>	structurerc<00><><00>g}dtj<00>jd<02><00><00>}|r|d|<04><00>z
}|dj<00>rJt	|<06>d<05>|d||ddzdt|d	<00>t|d
 								<00>|d<0B><04><0C>}|j
|<07>d
|d<00>d<0F>}|d|dd<00>d|dd<00>d<0F>z
}|d|dd<17>d<0F>z
}|d|drdn|drdnd<1D>d<1E>z
}t	|<06>d<1F>||d d!|d"<22><02><0C>}	|j
|	<09>|dj<00>rOd#|d<00>d$<24>}
 								|
 								d%|dd&d'<00>d(<28>z
}
 								t	|<06>d)<29>|
 								|d*d+t|d<00>|dd,<2C><03><0C>}|j
|<0B>|jj|<05>|S)-z-Generate meaning kernels from extracted data.<2E>kernel_z
%Y%m%d_%H%M%S<>_prq<00>_textrgY@<40>text_extractionrsrr)<04>type<70>
 								word_count<EFBFBD>
 								line_countr<EFBFBD>)rr
rrrzDiagram type: r<>z. zDimensions: r<>r<><00>xr<78>zAspect ratio: r<><00>.2fz
Orientation: r<><00>	landscaper<65><00>portrait<69>square<72>.<2E>
 								_structurer<EFBFBD>rn)r<>r<>zResearch diagram analysis: z with text content. zKey elements: N<><4E>r+<00>_summarygffffff<66>?<3F>summary)r<><00>text_length<74>structure_type)	rr<00>strftime<6D>striprrUr~r=rZ)rr<>r<>rrbr=<00>base_id<69>text_kernel<65>structure_content<6E>structure_kernelr<6C><00>summary_kernels            rrlz"DiagramProcessor._generate_kernelss><00><00>
 								<15><07><1C>H<EFBFBD>L<EFBFBD>L<EFBFBD>N<EFBFBD>3<>3<>O<EFBFBD>D<>E<>F<><07><13><13><12>H<EFBFBD>:<3A><EFBFBD>&<26>G<EFBFBD><15>[<5B>!<21>'<27>'<27>)<29>'<27>$<24>I<EFBFBD>U<EFBFBD>+<2B>!<21>+<2B>.<2E><1D>$<24>\<5C>2<>U<EFBFBD>:<3A>-<2D>"%<25>i<EFBFBD><07>&8<>"9<>"%<25>i<EFBFBD><07>&8<>"9<>!*<2A>	<12><0E>K<EFBFBD>
<14>N<EFBFBD>N<EFBFBD>;<3B>'<27>-<2D>Y<EFBFBD>7O<37>-P<>,Q<>QS<51>T<><19><19>|<7C>I<EFBFBD>l<EFBFBD>,C<>G<EFBFBD>,L<>+M<>Q<EFBFBD>y<EFBFBD>Ye<59>Of<4F>go<67>Op<4F>Nq<4E>qs<71>t<>t<><19><19>~<7E>i<EFBFBD><0E>.G<><03>-L<>B<EFBFBD>O<>O<><19><19>}<7D>I<EFBFBD>n<EFBFBD><U<>[<5B>ir<69>tA<02>jB<02>[e<>HP<02>-Q<02>QR<02>S<02>	S<02><19>(<28> <20>	<09><1A>,<2C>%<25><19><1A>,<2C>%<25><0E>
 								<EFBFBD><18>	<10><0E><0E>'<27>(<28><15>[<5B>!<21>'<27>'<27>)<29>3<>I<EFBFBD>>V<>4W<34>3X<33>Xl<58>m<>G<EFBFBD><13><1E>	<09>+<2B>(><3E>t<EFBFBD><03>(D<>'E<>S<EFBFBD>I<>I<>G<EFBFBD>*<2A>$<24>I<EFBFBD>X<EFBFBD>.<2E><1F><1D><1E>%<25>#&<26>y<EFBFBD><1B>'=<3D>#><3E>&/<2F>0H<30>&I<><12>
 								<0E>N<EFBFBD>
<14>N<EFBFBD>N<EFBFBD>><3E>*<2A>	
<0A><0C><0C><1B><1B>G<EFBFBD>$<24><16>rr=r\c	<00><><00>|std<01>y|dz}|D<00>cgc]}|j<00><00><02>}}t|d<04>5}tj||d<05><06>ddd<02>|dz}t|d<04>5}|jd<08>|jd	t
j<00>j<00><00>d
 								<EFBFBD><03>|jdt|<01><00>d<0C><03>|D]<5D>}|jd
|j<00>d
 								<EFBFBD><03>|jd|j<00>d
 								<EFBFBD><03>|jd|jd<10>d
 								<EFBFBD><03>|jd|j<00>d
 								<EFBFBD><03>|jd|j<00>d
 								<EFBFBD><03>|jd|j<00>d
 								<EFBFBD><03>|jdtj |j"d<05><06><00>d<0C><03><00><>	ddd<02>|dz}	t|	d<04>5}tj|j$|d<05><06>ddd<02>tdt|<01><00>d|<02><00><04>td|<03><00><02>td|<07><00><02>td|	<09><00><02>ycc}w#1swY<00><02>xYw#1swY<00><>xYw#1swY<00>lxYw)zSave kernels to files.zNo kernels to saveNzmeaning_kernels.json<6F>wr<77>)<01>indentzmeaning_kernels.mdz$# Meaning Kernels Extraction Report
 								zGenerated: <20>
 								zTotal kernels: z
 								z## Kernel: z- **Source**: z- **Confidence**: r<>z- **Timestamp**: z- **Hash**: z- **Content**: z- **Metadata**: zextraction_stats.jsonzSaved z kernels to z
 								  - JSON: z  - Markdown: z  - Statistics: )rPr(ri<00>json<6F>dump<6D>writerrrrUrrrrrr
<00>dumpsrr>)
 								rr=r\<00>	json_path<74>k<>kernels_data<74>f<>md_path<74>kernel<65>
 								stats_paths
 								          rr[zDiagramProcessor._save_kernelsTsQ<00><00><16><11>&<26>'<27><12> <20>"8<>8<>	<09>-4<>5<><01><01>	<09>	<09><0B>5<><0C>5<>
<11>)<29>S<EFBFBD>
!<21>	1<>Q<EFBFBD><10>I<EFBFBD>I<EFBFBD>l<EFBFBD>A<EFBFBD>a<EFBFBD>0<>	1<><1E> 4<>4<><07>
<11>'<27>3<EFBFBD>
<1F>	X<01>1<EFBFBD>
<0A>G<EFBFBD>G<EFBFBD>;<3B>=<3D>
<0A>G<EFBFBD>G<EFBFBD>k<EFBFBD>(<28>,<2C>,<2C>.<2E>":<3A>":<3A>"<<3C>!=<3D>R<EFBFBD>@<40>A<>
<0A>G<EFBFBD>G<EFBFBD>o<EFBFBD>c<EFBFBD>'<27>l<EFBFBD>^<5E>4<EFBFBD>8<>9<>!<21>
X<01><06><11><07><07>+<2B>f<EFBFBD>&6<>&6<>%7<>r<EFBFBD>:<3A>;<3B><11><07><07>.<2E><16><1D><1D><0F>r<EFBFBD>:<3A>;<3B><11><07><07>,<2C>V<EFBFBD>-><3E>-><3E>s<EFBFBD>,C<>2<EFBFBD>F<>G<><11><07><07>+<2B>F<EFBFBD>,<<3C>,<<3C>+=<3D>R<EFBFBD>@<40>A<><11><07><07>,<2C>v<EFBFBD>{<7B>{<7B>m<EFBFBD>2<EFBFBD>6<>7<><11><07><07>/<2F>&<26>.<2E>.<2E>)9<><12><<3C>=<3D><11><07><07>*<2A>4<EFBFBD>:<3A>:<3A>f<EFBFBD>o<EFBFBD>o<EFBFBD>a<EFBFBD>+P<>*Q<>QU<51>V<>W<>
X<01>	X<01>!<21>#:<3A>:<3A>
 								<EFBFBD>
<11>*<2A>c<EFBFBD>
"<22>	/<2F>a<EFBFBD><10>I<EFBFBD>I<EFBFBD>d<EFBFBD>j<EFBFBD>j<EFBFBD>!<21>A<EFBFBD>.<2E>	/<2F>	<0E><06>s<EFBFBD>7<EFBFBD>|<7C>n<EFBFBD>L<EFBFBD><1B>
<0A>><3E>?<3F>
<0A>
 								<EFBFBD>9<EFBFBD>+<2B>&<26>'<27>
<0A><0E>w<EFBFBD>i<EFBFBD>(<28>)<29>
<0A> <20><1A><0C>-<2D>.<2E><>;6<>	1<>	1<><31>
 									X<01>	X<01><> 	/<2F>	/<2F>s)<00>I<04>I<03>/EI&<03>$#I2<03>I#<07>&I/<07>2I;c<00>6<00>|jj<00>S)zGet processing statistics.)r><00>copyr's r<00>	get_statszDiagramProcessor.get_stats{s<00><00><13>z<EFBFBD>z<EFBFBD><EFBFBD><EFBFBD> <20> rr)r.r/r0r1rr2rrrrrer}rYr	rjr<>rkr<>rlrr[r<>r4rrr6r6GsI<00><00>B<>
 								<EFBFBD>t<EFBFBD>C<EFBFBD><13>H<EFBFBD>~<7E>
 								<EFBFBD>/<1B><13>/<1B>#<23>/<1B><14>m<EFBFBD>I\<5C>/<1B>b$<17>S<EFBFBD>$<17>C<EFBFBD>$<17>4<EFBFBD>P]<5D>K^<5E>$<17>L<19>e<EFBFBD>k<EFBFBD>k<EFBFBD><19>d<EFBFBD>3<EFBFBD><03>8<EFBFBD>n<EFBFBD><19>B <15>t<EFBFBD> <15><04>S<EFBFBD>	<09> <15>D<18>e<EFBFBD>k<EFBFBD>k<EFBFBD><18>d<EFBFBD>3<EFBFBD><03>8<EFBFBD>n<EFBFBD><18>(&<26>C<EFBFBD>&<26><13>&<26><13>&<26> *.<2E>C<17>4<EFBFBD><03>S<EFBFBD><08>><3E>C<17>$(<28><13>c<EFBFBD><18>N<EFBFBD>C<17>!$<24>C<17>$'<27>C<17>37<33>}<7D>2E<32>C<17>J%/<2F>T<EFBFBD>-<2D>%8<>%/<2F>t<EFBFBD>%/<2F>N!<21>4<EFBFBD><03>S<EFBFBD><08>><3E>!rr6c<00><00>tjd<01><02>}|jdd<04><05>|jddd<08><05>|jd	d
 								d<0B><05>|jdd
dd<0F><10>|j<00>}i}|jr3t|j<00>5}t
j|<03>}ddd<11>t|<02>}t|j<00>}|j<00>r<>|jj<00>dk(r|j||j<00>}<06>n2|jj<00>dvr|j!|<05>}<06>nt#d|j<00><00><02>t%j&d<15>n<>|j)<00>r<>g}|j+<00>D]<5D>}|jj<00>dk(r.|j||j<00>}|j-|<06><00>N|jj<00>dvs<01>k|j!|<08>}|j-|<06><00><>n#t#d|<05><00><02>t%j&d<15>|j/<00>}	t#d<17>t#d<18>t#d<19>t#d|	d<00><00><02>t#d|	d<00><00><02>t#d|	d<00><00><02>t#d |	d!<00><00><02>t#d<19>t%j&|	d!d"k(rd"<22>yd<15>y#1swY<00><02>LxYw)#z(Command line interface for the pipeline.z.Extract meaning kernels from research diagrams)<01>description<6F>inputz!Input PDF or image file/directory)<01>helpz-oz--outputzOutput directoryz-cz--configzConfiguration file (JSON)z-vz	--verbose<73>
 								store_truezVerbose output)<02>actionr<6E>Nz.pdf)rKz.jpgz.jpegz.tiffz.bmpzUnsupported file type: rGzInput not found: z3
 								==================================================zEXTRACTION SUMMARYz2==================================================zPages processed: r9zDiagrams found: r:zKernels extracted: r;zErrors: r<r)<18>argparse<73>ArgumentParser<65>add_argument<6E>
 								parse_argsr7rir<><00>loadr6rr<><00>is_file<6C>suffix<69>lowerre<00>outputrYrP<00>sys<79>exit<69>is_dir<69>iterdirrZr<>)
 								<EFBFBD>parser<65>argsr7r<><00>	processor<6F>
 								input_pathr=r_<00>	file_pathr>s
 								          r<00>mainr<6E>s<><00><00>
<15>
$<24>
$<24>1a<31>
b<>F<EFBFBD>
 								<EFBFBD><17><17><07>&I<><17>J<>
 								<EFBFBD><17><17><04>j<EFBFBD>/A<><17>B<>
 								<EFBFBD><17><17><04>j<EFBFBD>/J<><17>K<>
 								<EFBFBD><17><17><04>k<EFBFBD>,<2C>EU<45><17>V<><11><1C><1C><1E>D<EFBFBD><10>F<EFBFBD><0B>{<7B>{<7B>
<11>$<24>+<2B>+<2B>
<1E>	"<22>!<21><19>Y<EFBFBD>Y<EFBFBD>q<EFBFBD>\<5C>F<EFBFBD>	"<22>!<21><16>(<28>I<EFBFBD><16>d<EFBFBD>j<EFBFBD>j<EFBFBD>!<21>J<EFBFBD><11><19><19><1B><15><1C><1C>"<22>"<22>$<24><06>.<2E><1F>0<>0<><1A>T<EFBFBD>[<5B>[<5B>I<>G<EFBFBD>
<17>
<1E>
<1E>
$<24>
$<24>
&<26>*T<>
T<><1F>2<>2<>:<3A>><3E>G<EFBFBD><11>+<2B>J<EFBFBD>,=<3D>,=<3D>+><3E>?<3F>@<40><0F>H<EFBFBD>H<EFBFBD>Q<EFBFBD>K<EFBFBD>	<13>	<1A>	<1A>	<1C><18><0B>#<23>+<2B>+<2B>-<2D>	,<2C>I<EFBFBD><18><1F><1F>%<25>%<25>'<27>6<EFBFBD>1<>#<23>4<>4<>Y<EFBFBD><04><0B><0B>L<><07><1B>"<22>"<22>7<EFBFBD>+<2B><1A>!<21>!<21>'<27>'<27>)<29>-W<>W<>#<23>6<>6<>y<EFBFBD>A<><07><1B>"<22>"<22>7<EFBFBD>+<2B>
	,<2C>	<0E>!<21>*<2A><1C>.<2E>/<2F><0B><08><08><11><0B>
<16><1F><1F>!<21>E<EFBFBD>	<09>-<2D><18>	<09>
 								<1E><1F>	<09>&<26>M<EFBFBD>	<09><1D>e<EFBFBD>$5<>6<>7<>
 <EFBFBD>9<>	<09><1C>U<EFBFBD>#3<>4<>5<>
 <EFBFBD>7<>	<09><1F><05>&9<> :<3A>;<3B>
 								<<3C>=<3D>	<09>H<EFBFBD>U<EFBFBD>8<EFBFBD>_<EFBFBD>%<25>
 								&<26>'<27>	<09>&<26>M<EFBFBD><08>H<EFBFBD>H<EFBFBD>%<25><08>/<2F>Q<EFBFBD>&<26>Q<EFBFBD>.<2E>A<EFBFBD>.<2E>W	"<22>	"<22>s<00>K:<03>:L<07>__main__)r1<00>osr<73>r<>r<><00>pathlibrr<00>typingrrrrr <00>PILr	rhrMrPryrxrTrLrr6r<>r.r4rr<00><module>r<>s<><00><01><04>
 								<EFBFBD>
 								<EFBFBD><0B><0F><18><1D>,<2C>,<2C><0E>J<01><19><18>M<EFBFBD>
 								W<01><16><1E><17>
 								S<01><14><1E><17>
 								D<01>D<01>Bv!<21>v!<21>p	8/<2F>t<0C>z<EFBFBD><19><08>F<EFBFBD><1A><>K
<13>J<01><19>M<EFBFBD>	<09>
 								H<EFBFBD>I<>J<01><><13>W<01><1F><17>	<09>
 								U<EFBFBD>V<>W<01><><13>S<01><1F><17>	<09>
 								Q<EFBFBD>R<>S<01>s4<00>A+<00>B<00>B<00>+A=<03><A=<03>B<03>B<03>B'<03>&B'