热门标签 | HotTags
当前位置:  开发笔记 > 编程语言 > 正文

[寒江孤叶丶的Cocos2d-x之旅_36]用LUA实现UTF8的字符串基本操作UTF8字符串长度,UTF8字符串剪裁等

原创文章,欢迎转载,转载请注明:文章来自[寒江孤叶丶的Cocos2d-x之旅系列]博客地址:http:blog.csdn.netqq446569365一个用于UTF8字符串操作的

原创文章,欢迎转载,转载请注明:文章来自[寒江孤叶丶的Cocos2d-x之旅系列]

博客地址:http://blog.csdn.net/qq446569365

一个用于UTF8字符串操作的类。功能比較齐全,包含:

string.utf8len UTF8字符串长度

string.utf8sub 对UTF8字符串进行分割

string.utf8upper 大写转换

string.utf8lower 小写转换

string.utf8reverse 字符串逆转


Github下载地址: https://github.com/ArcherPeng/utf8String4LUA


--===========================--
--===========================--

-- $Id: utf8.lua 147 2007-01-04 00:57:00Z pasta $
--
-- Provides UTF-8 aware string functions implemented in pure lua:
-- * string.utf8len(s)
-- * string.utf8sub(s, i, j)
-- * string.utf8reverse(s)
--
-- If utf8data.lua (containing the lower<->upper case mappings) is loaded, these
-- additional functions are available:
-- * string.utf8upper(s)
-- * string.utf8lower(s)
--
-- All functions behave as their non UTF-8 aware counterparts with the exception
-- that UTF-8 characters are used instead of bytes for all units.

--[[
Copyright (c) 2006-2007, Kyle Smith
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of the author nor the names of its contributors may be
used to endorse or promote products derived from this software without
specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
--]]

-- ABNF from RFC 3629
--
-- UTF8-octets = *( UTF8-char )
-- UTF8-char = UTF8-1 / UTF8-2 / UTF8-3 / UTF8-4
-- UTF8-1 = %x00-7F
-- UTF8-2 = %xC2-DF UTF8-tail
-- UTF8-3 = %xE0 %xA0-BF UTF8-tail / %xE1-EC 2( UTF8-tail ) /
-- %xED %x80-9F UTF8-tail / %xEE-EF 2( UTF8-tail )
-- UTF8-4 = %xF0 %x90-BF 2( UTF8-tail ) / %xF1-F3 3( UTF8-tail ) /
-- %xF4 %x80-8F 2( UTF8-tail )
-- UTF8-tail = %x80-BF
--

-- returns the number of bytes used by the UTF-8 character at byte i in s
-- also doubles as a UTF-8 character validator
local function utf8charbytes (s, i)
-- argument defaults
i = i or 1

-- argument checking
if type(s) ~= "string" then
error("bad argument #1 to 'utf8charbytes' (string expected, got ".. type(s).. ")")
end
if type(i) ~= "number" then
error("bad argument #2 to 'utf8charbytes' (number expected, got ".. type(i).. ")")
end

local c = s:byte(i)

-- determine bytes needed for character, based on RFC 3629
-- validate byte 1
if c > 0 and c <= 127 then
-- UTF8-1
return 1

elseif c >= 194 and c <= 223 then
-- UTF8-2
local c2 = s:byte(i + 1)

if not c2 then
error("UTF-8 string terminated early")
end

-- validate byte 2
if c2 <128 or c2 > 191 then
error("Invalid UTF-8 character")
end

return 2

elseif c >= 224 and c <= 239 then
-- UTF8-3
local c2 = s:byte(i + 1)
local c3 = s:byte(i + 2)

if not c2 or not c3 then
error("UTF-8 string terminated early")
end

-- validate byte 2
if c == 224 and (c2 <160 or c2 > 191) then
error("Invalid UTF-8 character")
elseif c == 237 and (c2 <128 or c2 > 159) then
error("Invalid UTF-8 character")
elseif c2 <128 or c2 > 191 then
error("Invalid UTF-8 character")
end

-- validate byte 3
if c3 <128 or c3 > 191 then
error("Invalid UTF-8 character")
end

return 3

elseif c >= 240 and c <= 244 then
-- UTF8-4
local c2 = s:byte(i + 1)
local c3 = s:byte(i + 2)
local c4 = s:byte(i + 3)

if not c2 or not c3 or not c4 then
error("UTF-8 string terminated early")
end

-- validate byte 2
if c == 240 and (c2 <144 or c2 > 191) then
error("Invalid UTF-8 character")
elseif c == 244 and (c2 <128 or c2 > 143) then
error("Invalid UTF-8 character")
elseif c2 <128 or c2 > 191 then
error("Invalid UTF-8 character")
end

-- validate byte 3
if c3 <128 or c3 > 191 then
error("Invalid UTF-8 character")
end

-- validate byte 4
if c4 <128 or c4 > 191 then
error("Invalid UTF-8 character")
end

return 4

else
error("Invalid UTF-8 character")
end
end


-- returns the number of characters in a UTF-8 string
local function utf8len (s)
-- argument checking
if type(s) ~= "string" then
error("bad argument #1 to 'utf8len' (string expected, got ".. type(s).. ")")
end

local pos = 1
local bytes = s:len()
local len = 0

while pos <= bytes and len ~= chars do
local c = s:byte(pos)
len = len + 1

pos = pos + utf8charbytes(s, pos)
end

if chars ~= nil then
return pos - 1
end

return len
end

-- install in the string library
if not string.utf8len then
string.utf8len = utf8len
end


-- functions identically to string.sub except that i and j are UTF-8 characters
-- instead of bytes
local function utf8sub (s, i, j)
-- argument defaults
j = j or -1

-- argument checking
if type(s) ~= "string" then
error("bad argument #1 to 'utf8sub' (string expected, got ".. type(s).. ")")
end
if type(i) ~= "number" then
error("bad argument #2 to 'utf8sub' (number expected, got ".. type(i).. ")")
end
if type(j) ~= "number" then
error("bad argument #3 to 'utf8sub' (number expected, got ".. type(j).. ")")
end

local pos = 1
local bytes = s:len()
local len = 0

-- only set l if i or j is negative
local l = (i >= 0 and j >= 0) or s:utf8len()
local startChar = (i >= 0) and i or l + i + 1
local endChar = (j >= 0) and j or l + j + 1

-- can't have start before end!
if startChar > endChar then
return ""
end

-- byte offsets to pass to string.sub
local startByte, endByte = 1, bytes

while pos <= bytes do
len = len + 1

if len == startChar then
startByte = pos
end

pos = pos + utf8charbytes(s, pos)

if len == endChar then
endByte = pos - 1
break
end
end

return s:sub(startByte, endByte)
end

-- install in the string library
if not string.utf8sub then
string.utf8sub = utf8sub
end


-- replace UTF-8 characters based on a mapping table
local function utf8replace (s, mapping)
-- argument checking
if type(s) ~= "string" then
error("bad argument #1 to 'utf8replace' (string expected, got ".. type(s).. ")")
end
if type(mapping) ~= "table" then
error("bad argument #2 to 'utf8replace' (table expected, got ".. type(mapping).. ")")
end

local pos = 1
local bytes = s:len()
local charbytes
local newstr = ""

while pos <= bytes do
charbytes = utf8charbytes(s, pos)
local c = s:sub(pos, pos + charbytes - 1)

newstr = newstr .. (mapping[c] or c)

pos = pos + charbytes
end

return newstr
end


-- identical to string.upper except it knows about unicode simple case conversions
local function utf8upper (s)
return utf8replace(s, utf8_lc_uc)
end

-- install in the string library
if not string.utf8upper and utf8_lc_uc then
string.utf8upper = utf8upper
end


-- identical to string.lower except it knows about unicode simple case conversions
local function utf8lower (s)
return utf8replace(s, utf8_uc_lc)
end

-- install in the string library
if not string.utf8lower and utf8_uc_lc then
string.utf8lower = utf8lower
end


-- identical to string.reverse except that it supports UTF-8
local function utf8reverse (s)
-- argument checking
if type(s) ~= "string" then
error("bad argument #1 to 'utf8reverse' (string expected, got ".. type(s).. ")")
end

local bytes = s:len()
local pos = bytes
local charbytes
local newstr = ""

while pos > 0 do
c = s:byte(pos)
while c >= 128 and c <= 191 do
pos = pos - 1
c = s:byte(pos)
end

charbytes = utf8charbytes(s, pos)

newstr = newstr .. s:sub(pos, pos + charbytes - 1)

pos = pos - 1
end

return newstr
end

-- install in the string library
if not string.utf8reverse then
string.utf8reverse = utf8reverse
end



推荐阅读
  • 在C#编程中,数值结果的格式化展示是提高代码可读性和用户体验的重要手段。本文探讨了多种格式化方法和技巧,如使用格式说明符、自定义格式字符串等,以实现对数值结果的精确控制。通过实例演示,展示了如何灵活运用这些技术来满足不同的展示需求。 ... [详细]
  • 在软件开发过程中,经常需要将多个项目或模块进行集成和调试,尤其是当项目依赖于第三方开源库(如Cordova、CocoaPods)时。本文介绍了如何在Xcode中高效地进行多项目联合调试,分享了一些实用的技巧和最佳实践,帮助开发者解决常见的调试难题,提高开发效率。 ... [详细]
  • PHP预处理常量详解:如何定义与使用常量 ... [详细]
  • 零拷贝技术是提高I/O性能的重要手段,常用于Java NIO、Netty、Kafka等框架中。本文将详细解析零拷贝技术的原理及其应用。 ... [详细]
  • 本文详细介绍了 PHP 中对象的生命周期、内存管理和魔术方法的使用,包括对象的自动销毁、析构函数的作用以及各种魔术方法的具体应用场景。 ... [详细]
  • 解决问题:1、批量读取点云las数据2、点云数据读与写出3、csf滤波分类参考:https:github.comsuyunzzzCSF论文题目ÿ ... [详细]
  • 开机自启动的几种方式
    0x01快速自启动目录快速启动目录自启动方式源于Windows中的一个目录,这个目录一般叫启动或者Startup。位于该目录下的PE文件会在开机后进行自启动 ... [详细]
  • 在JavaWeb开发中,文件上传是一个常见的需求。无论是通过表单还是其他方式上传文件,都必须使用POST请求。前端部分通常采用HTML表单来实现文件选择和提交功能。后端则利用Apache Commons FileUpload库来处理上传的文件,该库提供了强大的文件解析和存储能力,能够高效地处理各种文件类型。此外,为了提高系统的安全性和稳定性,还需要对上传文件的大小、格式等进行严格的校验和限制。 ... [详细]
  • 大类|电阻器_使用Requests、Etree、BeautifulSoup、Pandas和Path库进行数据抓取与处理 | 将指定区域内容保存为HTML和Excel格式
    大类|电阻器_使用Requests、Etree、BeautifulSoup、Pandas和Path库进行数据抓取与处理 | 将指定区域内容保存为HTML和Excel格式 ... [详细]
  • 如何使用 `org.opencb.opencga.core.results.VariantQueryResult.getSource()` 方法及其代码示例详解 ... [详细]
  • 如何将TS文件转换为M3U8直播流:HLS与M3U8格式详解
    在视频传输领域,MP4虽然常见,但在直播场景中直接使用MP4格式存在诸多问题。例如,MP4文件的头部信息(如ftyp、moov)较大,导致初始加载时间较长,影响用户体验。相比之下,HLS(HTTP Live Streaming)协议及其M3U8格式更具优势。HLS通过将视频切分成多个小片段,并生成一个M3U8播放列表文件,实现低延迟和高稳定性。本文详细介绍了如何将TS文件转换为M3U8直播流,包括技术原理和具体操作步骤,帮助读者更好地理解和应用这一技术。 ... [详细]
  • Flowable 流程图路径与节点展示:已执行节点高亮红色标记,增强可视化效果
    在Flowable流程图中,通常仅显示当前节点,而路径则需自行获取。特别是在多次驳回的情况下,节点可能会出现混乱。本文重点探讨了如何准确地展示流程图效果,包括已结束的流程和正在执行的流程。具体实现方法包括生成带有高亮红色标记的图片,以增强可视化效果,确保用户能够清晰地了解每个节点的状态。 ... [详细]
  • Java Socket 关键参数详解与优化建议
    Java Socket 的 API 虽然被广泛使用,但其关键参数的用途却鲜为人知。本文详细解析了 Java Socket 中的重要参数,如 backlog 参数,它用于控制服务器等待连接请求的队列长度。此外,还探讨了其他参数如 SO_TIMEOUT、SO_REUSEADDR 等的配置方法及其对性能的影响,并提供了优化建议,帮助开发者提升网络通信的稳定性和效率。 ... [详细]
  • 在使用 Qt 进行 YUV420 图像渲染时,由于 Qt 本身不支持直接绘制 YUV 数据,因此需要借助 QOpenGLWidget 和 OpenGL 技术来实现。通过继承 QOpenGLWidget 类并重写其绘图方法,可以利用 GPU 的高效渲染能力,实现高质量的 YUV420 图像显示。此外,这种方法还能显著提高图像处理的性能和流畅性。 ... [详细]
  • 本文介绍了如何利用 Delphi 中的 IdTCPServer 和 IdTCPClient 控件实现高效的文件传输。这些控件在默认情况下采用阻塞模式,并且服务器端已经集成了多线程处理,能够支持任意大小的文件传输,无需担心数据包大小的限制。与传统的 ClientSocket 相比,Indy 控件提供了更为简洁和可靠的解决方案,特别适用于开发高性能的网络文件传输应用程序。 ... [详细]
author-avatar
955单车小宏
这个家伙很懒,什么也没留下!
PHP1.CN | 中国最专业的PHP中文社区 | DevBox开发工具箱 | json解析格式化 |PHP资讯 | PHP教程 | 数据库技术 | 服务器技术 | 前端开发技术 | PHP框架 | 开发工具 | 在线工具
Copyright © 1998 - 2020 PHP1.CN. All Rights Reserved | 京公网安备 11010802041100号 | 京ICP备19059560号-4 | PHP1.CN 第一PHP社区 版权所有