Uninote
Uninote
用户根目录
每日点滴

<!DOCTYPE html> <html><head><meta http-equiv='Content-Type' content='text/html; charset=utf8'><title>log reader/query/storage 日志查询工具</title> <script> var g_resRoot = '../../__res/'; var g_json_outline = '{"children":[{"children":[],"level":1,"lvNumber":"1","text":"log reader config"},{"children":[],"level":1,"lvNumber":"2","text":"log query 部署"},{"children":[],"level":1,"lvNumber":"3","text":"log query design"},{"children":[],"level":1,"lvNumber":"4","text":"每次变换reqnum,自动插入分隔符号"},{"children":[],"level":1,"lvNumber":"5","text":"日志内容读取限制"},{"children":[],"level":1,"lvNumber":"6","text":"记录上次的request id;下次刷新时只取新的。"},{"children":[],"level":1,"lvNumber":"7","text":"请求时间"},{"children":[],"level":1,"lvNumber":"8","text":"misc todo"},{"children":[],"level":1,"lvNumber":"9","text":"log_storage.php 代码分离"},{"children":[],"level":1,"lvNumber":"10","text":"查看时触发刷新日志"},{"children":[],"level":1,"lvNumber":"11","text":"分析时数据库出错要报错。"},{"children":[],"level":1,"lvNumber":"12","text":"三张表都换为北京时间"},{"children":[],"level":1,"lvNumber":"13","text":"post参数显示验证"},{"children":[],"level":1,"lvNumber":"14","text":"这里不要默认填时间"},{"children":[],"level":1,"lvNumber":"15","text":"日志内容最大高度设置"}],"level":0,"lvNumber":"","text":"log reader/query/storage 日志查询工具"}'; </script> <link rel='stylesheet' type='text/css' href='../../__res/common.css'><style>.s0{color:#000000;} .s1{color:#0000ff;} .s2{color:#000000;} .s3{color:#c80000;} .s4{color:#000000;} .s5{color:#c80000;} .s6{color:#c80000;} .s7{color:#c80000;} .s8{color:#000000;} .s9{color:#000000;} .s10{color:#0000ff;} .s11{color:#000000;} .s12{color:#000000;} .s13{color:#000000;} .s14{color:#000000;} .s15{color:#000000;} .s16{color:#000000;} </style> </head><body> <script src='../../__res/util.js'></script><div id='outline'> <li><a href='#ol_0' style='font-weight:bold; font-size:22px'>log reader/query/storage 日志查询工具</a></li> <li> <a href='#ol_1' >log reader config</a></li> <li> <a href='#ol_2' >log query 部署</a></li> <li> <a href='#ol_3' >log query design</a></li> <li> <a href='#ol_4' >每次变换reqnum,自动插入分隔符号</a></li> <li> <a href='#ol_5' >日志内容读取限制</a></li> <li> <a href='#ol_6' >记录上次的request id;下次刷新时只取新的。</a></li> <li> <a href='#ol_7' >请求时间</a></li> <li> <a href='#ol_8' >misc todo</a></li> <li> <a href='#ol_9' >log_storage.php 代码分离</a></li> <li> <a href='#ol_10' >查看时触发刷新日志</a></li> <li> <a href='#ol_11' >分析时数据库出错要报错。</a></li> <li> <a href='#ol_12' >三张表都换为北京时间</a></li> <li> <a href='#ol_13' >post参数显示验证</a></li> <li> <a href='#ol_14' >这里不要默认填时间</a></li> <li> <a href='#ol_15' >日志内容最大高度设置</a></li> </div>

<pre id='ol_0' class='title'>log reader/query/storage 日志查询工具</pre> <pre><d class='s0'><a href='log_ana.html#5346e0d5-0cd1-4cfa-9bf5-4f2fd02ce03a'>log parser/ana </a></d></pre> <pre><d class='s0'></d></pre> <pre><d class='s1'><a href='http://a.add.my/log_storage/'>http://a.add.my/log_storage/</a></d></pre> <pre><d class='s0'></d></pre> <pre><d class='s0'>访问 GUI 时触发:<a href='log_ana.html#7354bcbf-d875-4392-b90d-85785577553e'>ana.php 一次性分析处理所有日志 </a></d></pre> <pre><d class='s1'></d></pre> <t id='8d97b55a-901a-48c0-b882-f2420e2529f6'> <pre class='tnd_head'>lv1= lv2= type=</pre> <h1 id='ol_1'>log reader config</h1> <pre><d class='s2'></d></pre> <pre><d class='s3'>已经内联分析,查询时即执行 ana。</d></pre> <pre><d class='s2'></d></pre> <pre><d class='s2'>---------- local:</d></pre> <pre><d class='s4'><?php</d></pre> <pre><d class='s4'>$g_config["pwd"] = "root";</d></pre> <pre><d class='s4'></d></pre> <pre><d class='s4'>/日志解析路径/</d></pre> <pre><d class='s4'>$g_config['item'] = [</d></pre> <pre><d class='s4'> [</d></pre> <pre><d class='s4'> "api" => "mynote_log", //模块名称</d></pre> <pre><d class='s4'> "path" => "E:/uninote/mynote/basic/runtime/seaslog/default" //实际情况日志路径</d></pre> <pre><d class='s4'> ],</d></pre> <pre><d class='s4'> [</d></pre> <pre><d class='s4'> "api" => "mynote_login",</d></pre> <pre><d class='s4'> "path" => "E:/uninote/mynote/basic/runtime/seaslog/login"</d></pre> <pre><d class='s4'> ],</d></pre> <pre><d class='s4'> [</d></pre> <pre><d class='s4'> "api" => "mynote_request",</d></pre> <pre><d class='s4'> "path" => "E:/uninote/mynote/basic/runtime/logs/logs.txt"</d></pre> <pre><d class='s4'> ]</d></pre> <pre><d class='s4'>];</d></pre> <pre><d class='s2'></d></pre> <pre><d class='s2'>---------- online:</d></pre> <pre><d class='s2'></d></pre> <pre><d class='s2'><?php</d></pre> <pre><d class='s2'>$g_config["pwd"] = "root123.";</d></pre> <pre><d class='s2'></d></pre> <pre><d class='s2'>/日志解析路径/</d></pre> <pre><d class='s2'>$g_config['item'] = [</d></pre> <pre><d class='s2'> [</d></pre> <pre><d class='s2'> "api" => "mynote_log", //模块名称</d></pre> <pre><d class='s2'> "path" => "/home/www/mynote/basic/runtime/seaslog/default" //实际情况日志路径</d></pre> <pre><d class='s2'> ],</d></pre> <pre><d class='s2'> [</d></pre> <pre><d class='s2'> "api" => "mynote_login",</d></pre> <pre><d class='s2'> "path" => "/home/www/mynote/basic/runtime/seaslog/login"</d></pre> <pre><d class='s2'> ],</d></pre> <pre><d class='s2'> [</d></pre> <pre><d class='s2'> "api" => "mynote_request",</d></pre> <pre><d class='s2'> "path" => "/home/www/mynote/basic/runtime/logs/logs.txt"</d></pre> <pre><d class='s2'> ]</d></pre> <pre><d class='s2'>];</d></pre> </t> <pre><d class='s1'></d></pre> <t id='232ef357-0d09-4344-a541-aff474c7dc62'> <pre class='tnd_head'>lv1= lv2= type=</pre> <h1 id='ol_2'>log query 部署</h1> <pre><d class='s0'><a href='log_query.html#8d97b55a-901a-48c0-b882-f2420e2529f6'>log reader config </a></d></pre> <pre><d class='s2'>nginx config: E:\back\git\mynote\online\nginx\vhost\additional_item_82.conf</d></pre> <pre><d class='s2'>sql/.sql 初始化</d><d class='s1'></d></pre> </t> <pre><d class='s1'></d></pre> <t id='198975c0-137b-4310-8d57-ed6216d8414c'> <pre class='tnd_head'>lv1= lv2= type=</pre> <h1 id='ol_3'>log query design</h1> <pre><d class='s0'>记录、分析、查看</d></pre> <pre><d class='s5'>user mynote_login mynote_request mynote_log</d><d class='s2'></d></pre> <pre><d class='s1'></d></pre> <pre><d class='s0'>筛选(filter):按人、时间</d></pre> <pre><d class='s2'></d></pre> <pre><d class='s6'>request.id 显示在头部(红色),log.id 显示在每条日志中(绿色区域)</d></pre> <pre><d class='s6'>日志内容显示在蓝色区域</d></pre> <pre><d class='s7'>post参数显示在黄色区域,可以直接复制到api/tests,进行调试</d><d class='s2'></d></pre> <pre><d class='s2'></d></pre> <pre><d class='s2'>过滤条件(每张表可能都有过滤字段):</d></pre> <pre><d class='s2'>uid, 手机号、reqnum、时间段</d></pre> <pre><d class='s2'>message like</d></pre> <pre><d class='s2'>uri</d></pre> <pre><d class='s2'>请求消耗时间 > 200ms</d></pre> <pre><d class='s2'></d></pre> <pre><d class='s2'>要点:</d></pre> <pre><d class='s2'>查询的主要脉络应该是request表,但request可能不和任何user、login(无登录记录)、log(此request无任何日志输出)表关联</d></pre> <pre><d class='s2'></d></pre> <pre><d class='s2'>查询结果排序:</d></pre> <pre><d class='s2'>按request id排序,则无法按照日志发生的先后顺序显示;</d></pre> <pre><d class='s2'>按log id排序,因为有的request无log,因此此类request显示有问题。</d></pre> <pre><d class='s2'>解决方案:</d></pre> <t id='acca157d-9166-400f-a919-249957daa30e'> <pre class='tnd_head'>lv1= lv2= type=todo</pre> <pre><d class='s0'>cyb:使用日志时间排序</d></pre> <pre><d class='s0'>IFNULL(log.datetime,r.datetime) AS 记录时间,</d></pre> </t> <pre><d class='s2'></d></pre> </t> <pre><d class='s2'></d></pre> <t id='acaa91cf-802c-4ed7-8800-5c8d1b25b53e'> <pre class='tnd_head'>lv1=mynote lv2= type=todo</pre> <h1 id='ol_4'>每次变换reqnum,自动插入分隔符号</h1> <pre><d class='s4'>每个request的相同信息,都显示在此分隔符中:</d></pre> <pre><d class='s4'>user 相关、请求相关(时间、uri。。)</d></pre> <pre><d class='s4'>根据sessionid找user时,不能简单的做表关联,需要根据时间来找到唯一的用户,或者没有用户</d></pre> </t> <pre><d class='s1'></d></pre> <t id='492279b2-de36-497f-b3d9-b7204b6927a8'> <pre class='tnd_head'>lv1=mynote lv2= type=todo</pre> <h1 id='ol_5'>日志内容读取限制</h1> <pre><d class='s2'>不要超过700字节(可配置),超过点可以点击链接查看详细内容</d><d class='s1'></d></pre> </t> <pre><d class='s2'></d></pre> <t id='b783ba94-e557-4b77-be4b-0087cf3a643e'> <pre class='tnd_head'>lv1=mynote lv2= type=</pre> <h1 id='ol_6'>记录上次的request id;下次刷新时只取新的。</h1> </t> <pre><d class='s2'></d></pre> <t id='2d21bf17-8c08-4345-969d-ac252b2b86a9'> <pre class='tnd_head'>lv1=mynote lv2= type=</pre> <h1 id='ol_7'>请求时间</h1> <pre><d class='s0'>1、request表加入请求时间字段,同样毫秒级别;建立索引</d></pre> <pre><d class='s8'>2、不传参数,默认只查询30分钟内</d></pre> </t> <pre><d class='s2'></d></pre> <h1 id='ol_8'>misc todo</h1> <t id='71f49ef7-9eb8-4001-8892-7f67c14ed71b'> <pre class='tnd_head'>lv1= lv2= type=todo</pre> <pre><d class='s2'>多余行不要</d></pre> <pre><d class='s2'><!-- <tr>--></d></pre> <pre><d class='s2'><!-- <td colspan="--><?//= $field_num; ?><!--"></td>--></d></pre> <pre><d class='s2'><!-- </tr>--></d></pre> <pre><d class='s2'></d></pre> <pre><d class='s2'>api请求URL(->请求URL) 等字段名称优化</d></pre> <pre><d class='s0'></d></pre> <pre><d class='s0'>这个没有日志:</d></pre> <pre><d class='s0'><a href='http://dev.rongyipiao.com/link/take/lib/php/upload.php?guid=1535876647528'>http://dev.rongyipiao.com/link/take/lib/php/upload.php?guid=1535876647528</a></d></pre> </t> <pre><d class='s2'></d></pre> <t id='e9d489d6-c18b-41ae-b0c4-f8d3255f5e03'> <pre class='tnd_head'>lv1=mynote lv2= type=todo</pre> <h1 id='ol_9'>log_storage.php 代码分离</h1> <pre><d class='s4'>controller & view</d></pre> <pre><d class='s4'>view只是模板,只负责填数据,除了foreach,其他控制模块尽量都不使用</d></pre> <pre><d class='s2'>仿照db differ</d></pre> <pre><d class='s1'></d></pre> <pre><d class='s9'>从mynote移动到additional项目</d></pre> </t> <pre><d class='s2'></d></pre> <t id='9118860a-730c-4109-873c-6d789496c8b7'> <pre class='tnd_head'>lv1= lv2= type=todo</pre> <h1 id='ol_10'>查看时触发刷新日志</h1> <pre><d class='s0'><a href='log_ana.html#7354bcbf-d875-4392-b90d-85785577553e'>ana.php 一次性分析处理所有日志 </a></d></pre> </t> <pre><d class='s1'></d></pre> <t id='fa342af5-2381-43c8-82b6-a00f3add49d3'> <pre class='tnd_head'>lv1= lv2= type=todo</pre> <h1 id='ol_11'>分析时数据库出错要报错。</h1> <pre><d class='s9'>比如:insert失败(字段长度不够、类型不匹配、字段缺失)、连接出错</d><d class='s10'></d></pre> </t> <pre><d class='s0'></d></pre> <t id='c9dbffc9-c570-4921-a037-505c03acdc8f'> <pre class='tnd_head'>lv1= lv2= type=todo</pre> <h1 id='ol_12'>三张表都换为北京时间</h1> </t> <pre><d class='s2'></d></pre> <t id='ce514ae6-8c79-4ef2-868f-da605455a71b'> <pre class='tnd_head'>lv1= lv2= type=todo</pre> <h1 id='ol_13'>post参数显示验证</h1> <pre><d class='s9'>验证所有utf8编码都能够正常解析、显示</d><d class='s10'></d></pre> </t> <pre><d class='s2'></d></pre> <t id='b6ecf781-3502-41d0-ad42-e66f0b036037'> <pre class='tnd_head'>lv1= lv2= type=todo</pre> <h1 id='ol_14'>这里不要默认填时间</h1> <pre><d class='s9'>2 点击重置也没有效果</d><d class='s0'></d></pre> <pre><d class='s0'><img src='..\pic\aed849b99c2c7a4e6456a503bc991371.jpg' /></d><d class='s0'></d></pre> <pre><d class='s0'></d></pre> <pre><d class='s0'>手动删除时间后,外面的显示还有问题:</d></pre> <pre><d class='s0'><img src='..\pic\208b3cf63b8d13a7544d007e0383a3bd.jpg' /></d><d class='s0'></d></pre> <pre><d class='s0'></d></pre> <pre><d class='s0'>重置时恢复半小时</d></pre> </t> <pre><d class='s1'></d></pre> <t id='a6e32a75-c04c-4721-b9d3-e9961c3d32d4'> <pre class='tnd_head'>lv1= lv2= type=</pre> <h1 id='ol_15'>日志内容最大高度设置</h1> <pre><d class='s11'>配置修改,点击保存:修改到 </d><d class='s12'>url</d><d class='s11'>,并重新加载页面</d><d class='s13'></d></pre> <pre><d class='s14'></d></pre> <pre><d class='s14'>SHA-1: 7d0fa6905ea6588accc9f548299a909dfd7b73e4</d></pre> <pre><d class='s14'> </d><d class='s15'>日志内容最大高度(不设置则不限制高度)</d><d class='s14'></d></pre> <pre><d class='s16'></d></pre> </t> <pre><d class='s2'></d></pre> </body></html>

log_ana

log_viewer

点赞(0) 阅读(73) 举报
目录
标题