Skip to content

Commit b59e95d

Browse files
HugoFaraHugoFara
committed
Updates expression parsing with japanese to avoid HugoFara#69.
1 parent 79f9381 commit b59e95d

3 files changed

Lines changed: 74 additions & 37 deletions

File tree

‎docs/CHANGELOG.md‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -48,17 +48,19 @@ Solves [#129](https://github.com/HugoFara/lwt/issues/129), thanks to the help of
4848
* Text reading position was not working consistently when adding several known words.
4949
* Japanese was always requiring MeCab for TTS, even if it was not used
5050
([#155](https://github.com/HugoFara/lwt/pull/155)).
51-
* Fixes on multi-words:
51+
* Multi-words:
5252
* Tooltip was not properly saved ([#170](https://github.com/HugoFara/lwt/pull/170)).
5353
* Translation may be escaped two times ([#170](https://github.com/HugoFara/lwt/pull/170)).
5454
* Fixes [#170](https://github.com/HugoFara/lwt/pull/170): on creation,
5555
multi-word was always displayed before the text it encompasses,
5656
without removing this text.
5757
* Tootltip title was not properly saved, normally without incidence on user display.
5858
* Fixes [#69](https://github.com/HugoFara/lwt/issues/69): having the same
59-
multi-word on multiple texts was displaying it at many positions on creation.
60-
* Parsing Japanese texts with MeCab was creating warnings when the text was not finishing by a punctation mark.
61-
* Multi-words were not saved with MeCab parsing on Japanese.
59+
multi-word on multiple texts was displaying it at many positions on creation.
60+
* Japanese:
61+
* Parsing Japanese texts with MeCab was creating warnings when the text was not
62+
finishing by a punctation mark.
63+
* Multi-words were not saved with MeCab parsing on Japanese.
6264

6365
### Deprecated
6466

‎docs/info.html‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2235,7 +2235,7 @@ <h4>Fixed</h4>
22352235
<li>Text reading position was not working consistently when adding several known words.</li>
22362236
<li>Japanese was always requiring MeCab for TTS, even if it was not used
22372237
(<a href="https://github.com/HugoFara/lwt/pull/155">#155</a>).</li>
2238-
<li>Fixes on multi-words:
2238+
<li>Multi-words:
22392239
<ul>
22402240
<li>Tooltip was not properly saved (<a href="https://github.com/HugoFara/lwt/pull/170">#170</a>).</li>
22412241
<li>Translation may be escaped two times (<a href="https://github.com/HugoFara/lwt/pull/170">#170</a>).</li>
@@ -2247,9 +2247,14 @@ <h4>Fixed</h4>
22472247
multi-word on multiple texts was displaying it at many positions on creation.</li>
22482248
</ul>
22492249
</li>
2250-
<li>Parsing Japanese texts with MeCab was creating warnings when the text was not finishing by a punctation mark.</li>
2250+
<li>Japanese:
2251+
<ul>
2252+
<li>Parsing Japanese texts with MeCab was creating warnings when the text was not
2253+
finishing by a punctation mark.</li>
22512254
<li>Multi-words were not saved with MeCab parsing on Japanese.</li>
22522255
</ul>
2256+
</li>
2257+
</ul>
22532258
<h4>Deprecated</h4>
22542259
<ul>
22552260
<li>Using any function from <code>unloadformcheck.js</code> without using <code>lwt_form_check</code> is deprecated.</li>

‎inc/session_utility.php‎

Lines changed: 61 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -4143,13 +4143,11 @@ function getScriptDirectionTag($lid): string
41434143
*
41444144
* @return string[][] Append text and values to insert to the database
41454145
*
4146-
* @since 2.5.0-fork Function added.
4147-
*
41484146
* @global string $tbpref Table name prefix
41494147
*
41504148
* @psalm-return list{array<int, string>, list{0?: string,...}}
41514149
*/
4152-
function insert_expression_from_mecab($text, $lid, $wid, $len): array
4150+
function insertMecabExpression($text, $lid, $wid, $len): array
41534151
{
41544152
global $tbpref;
41554153

@@ -4183,6 +4181,9 @@ function insert_expression_from_mecab($text, $lid, $wid, $len): array
41834181
$sqlarray = array();
41844182
// For each sentence in database containing $text
41854183
while ($record = mysqli_fetch_assoc($res)) {
4184+
if (!array_key_exists((int) $record['SeTxID'], $mwords)) {
4185+
$mwords[(int) $record['SeTxID']] = array();
4186+
}
41864187
$sent = trim((string) $record['SeText']);
41874188
$fp = fopen($db_to_mecab, 'w');
41884189
fwrite($fp, $sent . "\n");
@@ -4213,9 +4214,9 @@ function insert_expression_from_mecab($text, $lid, $wid, $len): array
42134214
$sqlarray[] = "($wid, $lid, {$record['SeTxID']}, {$record['SeID']},
42144215
$pos, $len, " . convert_string_to_sqlsyntax_notrim_nonull($text) . ")";
42154216
if (getSettingZeroOrOne('showallwords', 1)) {
4216-
$mwords[$pos] = "&nbsp;$len&nbsp";
4217+
$mwords[(int) $record['SeTxID']][$pos] = "&nbsp;$len&nbsp";
42174218
} else {
4218-
$mwords[$pos] = $text;
4219+
$mwords[(int) $record['SeTxID']][$pos] = $text;
42194220
}
42204221
$seek = mb_strpos($parsed_sentence, $parsed_text, $seek + 1);
42214222
}
@@ -4227,6 +4228,33 @@ function insert_expression_from_mecab($text, $lid, $wid, $len): array
42274228
return array($mwords, $sqlarray);
42284229
}
42294230

4231+
/**
4232+
* Insert an expression to the database using MeCab.
4233+
*
4234+
* @param string $text Text to insert
4235+
* @param string|int $lid Language ID
4236+
* @param string|int $wid Word ID
4237+
* @param int $len Number of words in the expression
4238+
*
4239+
* @return string[][] Append text and values to insert to the database
4240+
*
4241+
* @since 2.5.0-fork Function added.
4242+
*
4243+
* @deprecated Since 2.10.0 Use insertMecabExpression
4244+
*
4245+
* @global string $tbpref Table name prefix
4246+
*
4247+
* @psalm-return list{array<int, string>, list{0?: string,...}}
4248+
*/
4249+
function insert_expression_from_mecab($text, $lid, $wid, $len): array
4250+
{
4251+
list($mwords, $sqlarr) = insertMecabExpression($text, $lid, $wid, $len);
4252+
$flat_mwords = array_reduce($mwords, function ($carry, $item) {
4253+
return $carry + $item;
4254+
}, []);
4255+
return array($flat_mwords, array(), $sqlarr);
4256+
}
4257+
42304258
/**
42314259
* Insert an expression to the database using MeCab.
42324260
*
@@ -4242,7 +4270,7 @@ function insert_expression_from_mecab($text, $lid, $wid, $len): array
42424270
* $mode is unnused, data are always returned.
42434271
* The second return argument is always empty array.
42444272
*
4245-
* @deprecated Use insert_expression_from_mecab instead.
4273+
* @deprecated Use insertMecabExpression instead.
42464274
*
42474275
* @global string $tbpref Table name prefix
42484276
*
@@ -4266,7 +4294,7 @@ function insertExpressionFromMeCab($textlc, $lid, $wid, $len, $mode): array
42664294
*
42674295
* @global string $tbpref Table name prefix
42684296
*/
4269-
function standardInsertMultiWord($textlc, $lid, $wid, $len): array
4297+
function insertStandardExpression($textlc, $lid, $wid, $len): array
42704298
{
42714299
global $tbpref;
42724300
$mwords = array();
@@ -4372,13 +4400,13 @@ function standardInsertMultiWord($textlc, $lid, $wid, $len): array
43724400
* @since 2.5.0-fork Mode is unnused and data are always added to the output.
43734401
* @since 2.5.2-fork Fixed multi-words insertion for languages using no space.
43744402
*
4375-
* @deprecated Since 2.10.0-fork, use standardInsertMultiWord
4403+
* @deprecated Since 2.10.0-fork, use insertStandardExpression
43764404
*
43774405
* @psalm-return list{array<int, null|string>, array<never, never>, list{0?: string,...}}
43784406
*/
43794407
function insert_standard_expression($textlc, $lid, $wid, $len, $mode): array
43804408
{
4381-
list($mwords, $sqlarr) = standardInsertMultiWord($textlc, $lid, $wid, $len);
4409+
list($mwords, $sqlarr) = insertStandardExpression($textlc, $lid, $wid, $len);
43824410
$flat_mwords = array_reduce($mwords, function ($carry, $item) {
43834411
return $carry + $item;
43844412
}, []);
@@ -4525,29 +4553,31 @@ function newMultiWordInteractable($hex, $multiwords, $wid, $len): void
45254553

45264554
?>
45274555
<script type="text/javascript">
4528-
let term = <?php echo json_encode($attrs); ?>;
4556+
(function () {
4557+
let term = <?php echo json_encode($attrs); ?>;
45294558

4530-
const multiWords = <?php echo json_encode($multiwords); ?>;
4559+
const multiWords = <?php echo json_encode($multiwords); ?>;
45314560

4532-
let title = '';
4533-
if (window.parent.LWT_DATA.settings.jQuery_tooltip) {
4534-
title = make_tooltip(
4535-
multiWords[window.parent.LWT_DATA.text.id][0], term.data_trans,
4536-
term.data_rom, parseInt(term.data_status, 10)
4561+
let title = '';
4562+
if (window.parent.LWT_DATA.settings.jQuery_tooltip) {
4563+
title = make_tooltip(
4564+
multiWords[window.parent.LWT_DATA.text.id][0], term.data_trans,
4565+
term.data_rom, parseInt(term.data_status, 10)
4566+
);
4567+
}
4568+
term['title'] = title;
4569+
let attrs = "";
4570+
Object.entries(term).forEach(([k, v]) => attrs += " " + k + '="' + v + '"');
4571+
// keys(term).map((k) => k + '="' + term[k] + '"').join(" ");
4572+
4573+
newExpressionInteractable(
4574+
multiWords[window.parent.LWT_DATA.text.id],
4575+
attrs,
4576+
term.data_code,
4577+
<?php echo json_encode($hex); ?>,
4578+
<?php echo json_encode($showAll); ?>
45374579
);
4538-
}
4539-
term['title'] = title;
4540-
let attrs = "";
4541-
Object.entries(term).forEach(([k, v]) => attrs += " " + k + '="' + v + '"');
4542-
// keys(term).map((k) => k + '="' + term[k] + '"').join(" ");
4543-
4544-
newExpressionInteractable(
4545-
multiWords[window.parent.LWT_DATA.text.id],
4546-
attrs,
4547-
term.data_code,
4548-
<?php echo json_encode($hex); ?>,
4549-
<?php echo json_encode($showAll); ?>
4550-
);
4580+
})()
45514581
</script>
45524582
<?php
45534583
flush();
@@ -4583,11 +4613,11 @@ function insertExpressions($textlc, $lid, $wid, $len, $mode): null|string
45834613
* Should separate the two
45844614
*/
45854615
if ('MECAB' == strtoupper(trim($regexp))) {
4586-
list($appendtext, $sqlarr) = insert_expression_from_mecab(
4616+
list($appendtext, $sqlarr) = insertMecabExpression(
45874617
$textlc, $lid, $wid, $len
45884618
);
45894619
} else {
4590-
list($appendtext, $sqlarr) = standardInsertMultiWord(
4620+
list($appendtext, $sqlarr) = insertStandardExpression(
45914621
$textlc, $lid, $wid, $len
45924622
);
45934623
}

0 commit comments

Comments
 (0)